Information processing device, information processing method, and program

ABSTRACT

In an image including a continuous visual field of 360 degrees, such as a celestial sphere image, in a case where a partial region is displayed as an output target region, transition from a certain output target region to the other output target region is realized smoothly and with natural feeling, without having visually uncomfortable feeling. For this reason, a transition source output target region and a transition destination output target region are specified, in an output target region which is a partial region of the entire image, which is an image having a continuous visual field of 360 degrees in at least one direction. Then, a visual field transition path from the specified transition source output target region to the specified transition destination output target region is automatically determined.

TECHNICAL FIELD

The present technology relates to an information processing device, aninformation processing method, and a program, and in particular, relatesto a technology used at the time of displaying a partial region as anoutput target region with respect to an image having a continuous visualfield of 360 degrees.

BACKGROUND ART

In image contents as a still image or a moving image captured by animage pickup device, image contents produced by computer graphics, orthe like, an image having a continuous visual field of 360 degrees hasbeen known. Examples of the image contents include a so-called celestialsphere image, a half celestial sphere image, a panorama image of 360degrees, and the like.

In a case where such an image having a continuous visual field of 360degrees is displayed on a display device, a part of the entire image iscut out as an output target region and is displayed.

In Patent Literature 1, it is described that a part of the entire imageis cut out and is displayed such that a specific subject is tracked, atthe time of viewing an image having a wide field angle, for example, acelestial sphere image or the like.

CITATION LIST Patent Literature

Patent Literature 1: JP-A-2014-220724

DISCLOSURE OF INVENTION Technical Problem

However, in the celestial sphere image or the like, when a display imageis transitioned to the other output target region at the time ofdisplaying a part of the output target region, it is required to performtransition desirable for an audience. This is because in a case wherethe output target region is needlessly changed on the display when theaudience views a part of an image cut out from the image celestialsphere image or the like, the audience may feel that the image is anunnatural image or an image having an uncomfortable feeling on thebehavior of the subject.

Therefore, an object of the present technology is to provide an imagewhich is comfortable and is easily visible for an audience, and is notunnatural, even at the time of transitioning an output target region.

Solution to Problem

An information processing device according to the present technology,includes: a specification unit that specifies a transition source outputtarget region and a transition destination output target region, as anoutput target region which is a partial region of an entire image, whichis an image including a continuous visual field of 360 degrees in atleast one direction; and a visual field transition path determinationunit that automatically determines a visual field transition path fromthe transition source output target region to the transition destinationoutput target region.

In the transition of the output target region imitating a viewpointmovement of an audience (a region cut out from the entire image, and isset to a display target), in a case of assuming a transition source anda transition destination in the entire continuous image of 360 degrees,the visual field transition path from the transition source outputtarget region to the transition destination output target region as acontinuous image can be variously considered, and thus, the visual fieldtransition path is automatically determined.

Note that, the entire image is an image including at least a continuousportion of 360 degrees, such as a celestial sphere image, a halfcelestial sphere image, and a panorama image of 360 degrees. In otherwords, the entire image is an image having a visual field of 360degrees.

In the information processing device according to the present technologydescribed above, it is considered that the visual field transition pathdetermination unit determines one of a candidate path progressing to onedirection from the transition source output target region towards thetransition destination output target region, and a candidate pathprogressing to a direction opposite to the one direction, as the visualfield transition path.

In the transition of the output target region, in a case of assuming thetransition source and the transition destination in the entirecontinuous image of 360 degrees, a plurality of paths transitioned fromthe transition source to the transition destination as the continuousimage are assumed. In particular, in consideration of a positionrelationship between the transition source and the transitiondestination in the entire continuous image of 360 degrees, it isconsidered that the transition from the transition source to thetransition destination (the viewpoint movement) is in a directionopposite to one direction on a circumference of 360 degrees of theentire image. One of the plurality of paths is set to be automaticallyselected. Therefore, in the candidate path progressing to one directionand the candidate path progressing to the direction opposite to onedirection, for example, a visually desired path is automaticallydetermined as the visual field transition path.

In the information processing device according to the present technologydescribed above, it is considered that the visual field transition pathdetermination unit uses movement distance information indicating amovement distance from the transition source output target region to thetransition destination output target region for each of a plurality ofcandidate paths, in order to determine the visual field transition path.

For example, a candidate path having a shorter movement distance isselected as the visual field transition path.

In the information processing device according to the present technologydescribed above, it is considered that the entire image is a movingimage, and the visual field transition path determination unit usesmovement direction information indicating a movement direction in theentire image of a subject existing in the transition source outputtarget region on progress of the moving image, in order to determine thevisual field transition path.

That is, in the transition source output target region, in a case wherea movable body other than a figure to be the subject, is moved in acertain direction on the moving image, the movement direction is onefactor for determining the visual field transition path.

In the information processing device according to the present technologydescribed above, it is considered that the entire image is a movingimage, and the visual field transition path determination unit usesmovement direction information indicating a movement direction in theentire image of a subject existing in the transition destination outputtarget region on progress of the moving image, in order to determine thevisual field transition path.

That is, in the transition destination output target region, in a casewhere the movable body other than the figure to be the subject is movedin a certain direction on the moving image, the movement direction isone factor for determining the visual field transition path.

In the information processing device according to the present technologydescribed above, it is considered that the entire image is a movingimage, and the visual field transition path determination unit usesmovement speed information indicating a movement speed in the entireimage of a subject existing in the transition destination output targetregion or a subject existing in the transition source output targetregion on progress of the moving image, in order to determine the visualfield transition path.

That is, in a case where one or both of images of each of the transitionsource and the transition destination (the movable body other than thefigure to be the subject) are moved on the moving image, the movementspeed is one factor for determining the visual field transition path.

In the information processing device according to the present technologydescribed above, it is considered that the entire image is a movingimage, and the visual field transition path determination unit usessubject position information indicating a position of a subject existingin the transition destination output target region in a frame after arequired transition time has elapsed, in order to determine the visualfield transition path.

In the continuous transition from the transition source to thetransition destination (the viewpoint movement), the transition time isrequired. In a case where the subject of the transition destination hasa motion, it is assumed that the position of the subject in the entireimage after the required transition time, is different from a positionin the frame at a transition start time point. Therefore, the visualfield transition path is determined in consideration of a subjectposition of the transition destination when the required transition timehas elapsed.

In the information processing device according to the present technologydescribed above, it is considered that the entire image is a movingimage, and the specification unit specifies the transition source outputtarget region and the transition destination output target region, on abasis of a selective instruction of a user with respect to a transitionsetting image representing a plurality of subject regions each of whichis a partial region included in the moving image and indicates anexistence position of each of a plurality of subjects in the movingimage, in a list.

The position of each of the subjects in the entire image at each timepoint is displayed in the list, with respect to the user who is a viewerof the image, and thus, for example, the subject can be selected on thetransition setting image as the list. Accordingly, the movement of theoutput target region, that is, the viewpoint movement is performedaccording to the progress of the moving image, and thus, it is possibleto present the subject that the user wants to view while performing theviewpoint movement.

In the information processing device according to the present technologydescribed above, it is considered that the specification unit specifiesthe transition destination output target region or the transition sourceoutput target region, on a basis of a user manipulation of designating adisplay candidate image, that is performed with respect to a displaybased on display data including an image of a partial region to be theoutput target region in the entire image and an existence presentationimage representing a display candidate image existing in a still imageor a moving image as the entire image.

The existence of the subject of the other display candidate is presentedas the existence presentation image, on the displayed image. Then, theuser performs a manipulation of selecting a certain display candidateimage by the manipulation or the like with respect to the existencepresentation image, and thus, image display of performing the viewpointmovement to a region where the display candidate image is included, isrealized.

In the information processing device according to the present technologydescribed above, it is considered that the existence presentation imageis a thumbnail image of the display candidate image, and the displaydata includes an image of a current output target region and a thumbnailimage of the display candidate image.

That is, existence presentation is performed by a method ofsuperimposing the existing display candidate image, for example, on apart of the current output target region, as a thumbnail image having auniform size.

In the information processing device according to the present technologydescribed above, it is considered that the existence presentation imageis provided at a position corresponding to a position relationship ofthe display candidate image indicated by the existence presentationimage with respect to the image of the partial region.

For example, the existence presentation image of the display candidateimage existing in an upper portion on a screen with respect to thecurrent output target region, is presented in the upper portion on thecurrent screen. The existence presentation image of the displaycandidate image existing in a right portion on the screen with respectto the current output target region, is presented in the right portionon the current screen.

In the information processing device according to the present technologydescribed above, it is considered that the display data includes anexistence presentation image corresponding to a display candidate imageexisting in a designated time axis position, that corresponds to thetime axis position designated in the image of the partial region to bethe output target region.

In a case of moving image contents, an existence state of each of thedisplay candidate images varies even in a time axis direction.Therefore, it is possible to confirm the existence of the displaycandidate image on the time axis.

In the information processing device according to the present technologydescribed above, it is considered that the display data includes anexistence presentation image indicating a position relationship betweenthe display candidate image and the image of the partial region to bethe output target region.

For example, an image indicating a relative position of each of thedisplay candidate images with respect to the current output targetregion is set to the existence presentation image.

An information processing method according to the present technology,includes: a specification step of specifying a transition source outputtarget region and a transition destination output target region, in anoutput target region which is a partial region of an entire image, whichis an image including a continuous visual field of 360 degrees in atleast one direction; and a visual field transition path determinationstep of automatically determining a visual field transition path fromthe transition source output target region to the transition destinationoutput target region.

Accordingly, the information processing device automatically determinesthe visual field transition path.

A program according to the present technology is a program that causesan information processing device to execute: the specification stepdescribed above; and the visual field transition path determination stepdescribed above. Accordingly, the information processing device capableof automatically determining the visual field transition path, isrealized.

Advantageous Effects of Invention

According to the present technology, it is possible to provide visuallysmooth transition to a viewer, on a display imitating a viewpointmovement as transition from a transition source output target region toa transition destination output target region.

Note that, the effects described herein are not necessarily limited, andmay be any effect described in the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory diagram of a visual field of an image to bedisplayed in an embodiment of the present technology.

FIG. 2 is an explanatory diagram of transition of an output targetregion of the embodiment.

FIG. 3 is a block diagram of an information processing device andperipheral devices of the embodiment.

FIG. 4 is a block diagram of a hardware configuration of the informationprocessing device of the embodiment.

FIG. 5 is a block diagram of a functional configuration of theinformation processing device of the embodiment.

FIG. 6 is an explanatory diagram of visual field transition pathdetermination using movement distance information of the embodiment.

FIG. 7 is an explanatory diagram of the output target region during thetransition of the embodiment.

FIG. 8 is an explanatory diagram of a visual line direction ofperforming interpolation on a spherical surface of the embodiment.

FIG. 9 is an explanatory diagram of visual field transition pathdetermination using movement direction information of a transitionsource of the embodiment.

FIG. 10 is an explanatory diagram of visual field transition pathdetermination using movement direction information of a transitiondestination of the embodiment.

FIG. 11 is an explanatory diagram of visual field transition pathdetermination using subject position information of the transitiondestination of the embodiment.

FIG. 12 is an explanatory diagram of an aspect of the transition of theoutput target region of the embodiment.

FIG. 13 is a flowchart of path determination processings I and II of theembodiment.

FIG. 14 is a flowchart of path determination processings III and IV ofthe embodiment.

FIG. 15 is a flowchart of path determination processing V of theembodiment.

FIG. 16 is an explanatory diagram of a transition setting image of theembodiment.

FIG. 17 is an explanatory diagram of visual field transition pathdetermination based on the transition setting image of the embodiment.

FIG. 18 is an explanatory diagram of the visual field transition pathdetermination based on the transition setting image of the embodiment.

FIG. 19 is a flowchart of processing using the transition setting imageof the embodiment.

FIG. 20 is an explanatory diagram of interpolation of three or morepoints of the embodiment.

FIG. 21 is an explanatory diagram of designation input using anexistence presentation image of the embodiment.

FIG. 22 is an explanatory diagram of the designation input using theexistence presentation image of the embodiment.

FIG. 23 is an explanatory diagram of the designation input using theexistence presentation image of the embodiment.

FIG. 24 is a flowchart of processing including displaying of theexistence presentation image of the embodiment.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, an embodiment will be described in the following order.

<1. Image of Processing Target of Embodiment>

<2. Configuration of Information Processing Device>

<3. Transition Processing of Output Target Region>

<4. Processing Using Transition Setting Image>

<5. Processing Using Existence Presentation Image>

<6. Conclusion and Modification Example>

1. Image of Processing Target of Embodiment

First, an image which is a processing target of an informationprocessing device of this embodiment will be described. The image to bethe processing target is an image having a continuous visual field of360 degrees in at least one direction. In other words, the image to bethe processing target is an image having a visual field of 360 degrees.One direction is a linear direction. A straight line as a direction on aspherical surface (a straight line in a case of ignoring a curvegenerated on a spherical surface in three-dimension) is also included.

A half celestial sphere image of FIG. 1A, a celestial sphere image ofFIG. 1B, a panorama image of 360 degrees of FIG. 1C, and the like areexemplified as a specific example of such an image.

In FIG. 1A, P0 is a reference position of a visual line. P0 can beregarded as a position of an image pickup device at the time ofcapturing an image, or can be regarded as a position of a viewer at thetime of displaying an image.

In the position P0, for example, in a case where capturing is performedby directing a wide-angle lens of the image pickup device, such as afisheye lens, towards a directly upper side, as illustrated in FIG. 1A,it is possible to obtain a still image or a moving image having anoptical axis L1 illustrated by a dot-and-dash line and a half celestialspherical range of which vertical points intersect with each other. Thatis, the still image or the moving image is a half celestial sphere imagehaving a field angle of 360 degrees in a horizontal direction and 180degrees in a vertical direction.

In addition, in the position P0 of FIG. 1B, for example, in a case wherecapturing is performed by disposing the wide-angle lens of the imagepickup device towards a directly upper side and a directly lower side,it is possible to obtain an image having a half celestial sphericalrange on the optical axis L1 side directed towards the directly upperside illustrated by a dot-and-dash line, and similarly, a half celestialspherical range on the optical axis L2 side directed towards thedirectly lower side illustrated by a dot-and-dash line. Then, in a casewhere an image captured through a wide-angle lens on an upper surfaceand an image captured through a wide-angle lens on a lower surface aresynthesized, a still image is generated in which one celestial sphericalrange is captured. In addition, in a case of capturing the moving image,in the image captured through the wide-angle lens on the upper surfaceand the image captured through the wide-angle lens on the lower surface,simultaneously captured frames are synthesized, and thus, a moving imageis generated in which a celestial spherical range is captured.

Such a still image or a moving image is a celestial sphere image havinga field angle of 360 degrees in the horizontal direction and 360 degreesin the vertical direction.

FIG. 1C is a so-called panorama image of 360 degrees. The image pickupdevice captures a range of 360 degrees around the image pickup device,and thus, it is possible to obtain a still image or a moving image asthe panorama image of 360 degrees. That is, the still image or themoving image is an image having a field angle of 360 degrees in thehorizontal direction.

Note that, here, an example of the field angle in the horizontaldirection is exemplified, but a panorama image of 360 degrees having afield angle of 360 degrees in the vertical direction is also assumed.

In this embodiment, control for display output or the like is performedby using such a continuous image having a visual field of 360 degrees inat least one direction as a processing target.

Hereinafter, the embodiment will be described by using the celestialsphere image of FIG. 1B as an example, but the processing to bedescribed can be performed by applying the half celestial sphere imageor the panorama image of 360 degrees as the processing target.

When the celestial sphere image is displayed on a display device, theinformation processing device of the embodiment is capable of cuttingout an output target region which is a partial region of the entireimage of the celestial sphere image (that is, the entire celestialsphere image having a field angle of 360 degrees in both directionsincluding the horizontal direction and the vertical direction) to bedisplayed.

FIG. 2A illustrates an aspect of cutting out a region for display in thecelestial sphere image. The entire image HP as the celestial sphereimage can be considered as an image of the entire inside circumferentialsurface of a sphere. That is, the entire image HP is the entire imagecapable of being seen from the position P0.

A region of a part of the entire image HP is cut out, and is set to anoutput target region 90. The output target region 90 illustrated by asolid line is an image viewed by the viewer in the position P0 in avisual line direction E1. For example, the image is displayed on thedisplay device by using a part of a region in a certain visual linedirection as the output target region 90.

Note that, here, an example of cutting out the region for display cutout has been described, but the region is not limited for display, andit is assumed that a part of a region is cut out, and is edited orrecorded in order to prepare edited contents. In such a case, theprocessing of this embodiment described below, in particular, processingrelevant to determination of a transition path can be applied.

The output target region 90 can be arbitrarily changed. For example, theoutput target region 90 can be an image of a region visible in a visualline direction E2, as illustrated by a broken line.

Thus, the output target region 90 can be selected as an arbitraryregion, in the entire image HP. Accordingly, in a case of a still image,the output target region 90 can be changed according to a usermanipulation, or the output target region 90 can be automaticallychanged by the information processing device.

In addition, in a case of a moving image, the output target region 90can be selected according to the user manipulation or the automaticcontrol, or the output target region 90 can be changed according to theprogress of the moving image.

Even though it is not illustrated, the size, an aspect ratio, or thelike of the output target region 90 can be set or changed according tothe user manipulation or the automatic control.

In addition, it is assumed that the output target region 90, forexample, is set to a region including a figure, an animal, a designatedtarget, or the like in the entire image HP as a subject.

For example, when a figure A and a figure B are included as the subjectin the entire image HP, processing is performed in which initially, aregion including the figure A is cut out as the output target region 90,and is displayed on the display device, and after that, the outputtarget region 90 is changed to a region including the figure B, and theregion including the figure B is cut out and is displayed on the displaydevice.

When such an output target region 90 is transitioned, the informationprocessing device of the embodiment performs visual field transitionpath determination processing.

FIG. 2B illustrates subjects 100 and 101. In a case where the outputtarget region 90 is transitioned from a region including the subject 100to a region including the subject 101, there are many cases where it isunnatural to suddenly switch a display screen. Therefore, a region to bedisplayed is gradually changed from the region including the subject 100to the region including the subject 101.

In a case of a celestial sphere image, paths PT1 and PT2 are assumed ina case where a straight line (in this case, as described above, a curveas a spherical surface is ignored) is considered as a visual fieldtransition path at the time of such transition.

The information processing device of this embodiment performs visualfield transition path determination in a plurality of paths (candidatepaths) such that transition comfortable for the viewer can be realized.For example, one of the paths PT1 and PT2 is selected as the visualfield transition path.

2. Configuration of Information Processing Device

The information processing device of this embodiment is embedded in adevice such as an image pickup device, an image display device, an imageediting device, and a display control device, or is realized by acomputer device or the like.

FIG. 3 illustrates an information processing device 1 of the embodiment,and an image pickup device 2, a storage device 3, a communication device4, a manipulation device 5, and a display device 6, as a peripheralconfiguration.

There is a case where the information processing device 1 is configuredas an integrated device along with all or a part of the image pickupdevice 2, the storage device 3, the communication device 4, themanipulation device 5, and the display device 6, illustrated as a deviceof each of the peripheral units. For example, there is a case where theinformation processing device 1 is integrated as a camera, a monitordisplay device, a television device, an image editing device, a computerdevice, a terminal device, or the like.

In addition, there is a case where the information processing device 1is a device separated from the illustrated peripheral devices, andperforms communication with respect to each of the peripheral devicesthrough a wired or wireless communication path.

The image pickup device 2 illustrated in FIG. 3 performs image capturingas a still image or a moving image, and outputs image data obtained bycapturing.

For example, the image pickup device 2 condenses light from the subjecton a capturing element such as a complementary metal oxide semiconductor(CMOS) type capturing element or a charge coupled device (CCD) typecapturing element, by an optical system provided with a lens such as acover lens, a zoom lens, and a focus lens, or a diaphragm mechanism.

It is assumed that the image pickup device 2 includes a wide-angle lensin the optical system, in order to capture an image having a continuousvisual field of 360 degrees. For example, it is assumed that the imagepickup device 2 includes a fisheye lens, and a 360 degrees lens. Inaddition, the image pickup device 2 may include an omnidirectionalcapturing optical system or an entire circumferential capturing opticalsystem, or may be configured by being attached with an adaptor includinga detachable omnidirectional capturing optical system or an entirecircumferential capturing optical system. It is considered that anambient reflection mirror or a panoramic annular lens (PAL) is used asthe omnidirectional capturing optical system or the entirecircumferential capturing optical system.

Then, in the image pickup device 2, an electric signal obtained byphotoelectric conversion of the capturing element, for example, issubjected to correlated double sampling (CDS) processing, automatic gaincontrol (AGC) processing, or the like, and is further subjected toanalog/digital (A/D) conversion processing, with respect to subjectlight condensed on the capturing element by such an optical systemlight. Then, a capturing signal as digital data, is subjected to clampprocessing of clamping a black level of R, G, and B to a predeterminedlevel, correction processing between color channels of R, G, and B, andprocessing of generating (separating) a brightness (Y) signal and acolor (C) signal from image data of R, G, and B. In addition, the imagedata subjected to such various signal processings is subjected todefinition conversion processing, or for example, encoding processingfor recording or communication.

Then, the image pickup device 2 outputs the image data generated by theencoding processing or image data before encoding (so-called, RAW imagedata) to the information processing device 1.

The storage device 3 stores the image data in a storage medium, orsupplies the image data read out from the storage medium to theinformation processing device 1, according to a request from theinformation processing device 1.

The storage device 3 stores an image file such as still image data ormoving image data, attribute information of the image file, a thumbnailimage, and the like, for example, by using a non-volatile memory as thestorage medium.

The image file, for example, is stored in the format of a jointphotographic experts group (JPEG), a tagged image file format (TIFF), agraphics interchange format (GIF), moving picture experts group(MPEG)-2, MPEG-4, an audio video interleave (AVI), advanced video codechigh definition (AVCHD: Registered Trademark), and the like.

In addition, there is a case where the storage device 3 records the RAWimage data as the still image data or the moving image data.

The actual configuration of the storage device 3 is variouslyconsidered. For example, the storage device 3 may be a storage deviceusing a flash memory as the storage medium, or may be configured of amemory card detachable with respect to a device including theinformation processing device 1 (for example, a transportable flashmemory) and a card recording/reproducing unit performingrecording/reproducing access with respect to the memory card. Inaddition, the storage device 3 is realized as a hard disk drive (HDD) orthe like. It is obvious that the storage device 3 may be arecording/reproducing unit corresponding to various storage mediums,such as an optical disk, a magnetooptical disk, a hologram memorymedium, a magnetic tape, an optical tape, and a volume recording medium.

The communication device 4 performs wired or wireless communication withrespect to the external device (not illustrated), according to therequest from the information processing device 1. For example, thecommunication device 4 transmits or receives the image data or controldata. Accordingly, the information processing device 1 is capable oftransmitting the image data or the control data to the external device(not illustrated), or of performing processing of acquiring the imagedata which is supplied from the external device.

It is considered that the communication device 4, for example, performscommunication according to a communication method of wirelesscommunication standard or the like, such as wireless fidelity (WIFI) orBluetooth.

In addition, the communication device 4 may perform communication as anetwork communication unit, for example, through various networks suchas the Internet, a home network, and a local area network (LAN), and maytransmit and receive various data items with respect to a server, aterminal, or the like on the network.

The manipulation device 5 has an input function of inputting the usermanipulation, and transmit a signal according to the input manipulationto the information processing device 1.

The manipulation device 5, for example, is realized as variousmanipulators provided on a housing of a device, a touch pad, a touchpanel formed in the display unit 34, and the like.

In addition, various manipulations can be performed according to a touchpanel manipulation using an icon, a menu, or the like displayed on thetouch panel and display device 6.

Alternatively, a tap manipulation of the user may be detected by thetouch pad.

In addition, a so-called gesture manipulation of the user may berecognized according to peripheral image recognition, or a soundmanipulation may be recognized by having a sound input function.

The display device 6 performs various displays with respect to the user.The display device 6, for example, includes display unit such as aliquid crystal display (LCD) or an organic electro-luminescence (EL)display, and a display driving device thereof.

In addition, the display device 6 receives the image data supplied fromthe output target region 90, for example, cut out from the celestialsphere image by the information processing device 1, and displays amoving image or a still image. In addition, the display device 6 executedisplay as various manipulation menus, icons, and messages, that is, agraphical user interface (GUI), on a screen, on the basis of aninstruction of the information processing device 1.

The information processing device 1 includes a central processing unit(CPU), a read only memory (ROM), a random access memory (RAM), and thelike.

In a case where the information processing device 1 is a deviceintegrated with all or a part of each of the peripheral units such asthe image pickup device 2, the storage device 3, the communicationdevice 4, the manipulation device 5, and the display device 6, theinformation processing device 1 functions as a control unit, andperforms operation control with respect to each of the peripheral unitsaccording to an operation state, the user manipulation, or the like, onthe basis of a program.

In a case where the information processing device 1 is a deviceseparated from each of the peripheral units, the information processingdevice 1 performs communication of the image data or the control datawith respect to the device of each of the peripheral units.

In this embodiment, the information processing device 1 performs atleast processing of determining the visual field transition path of theoutput target region 90 with respect to an image having a visual fieldof 360 degrees, such as a celestial sphere image.

The information processing device 1 performs the visual field transitionpath determination processing of the output target region 90 by usingthe image data captured by the image pickup device 2, the image dataread out from the storage medium by the storage device 3, or the imagedata received through the communication device 4 by being transmittedfrom the external device, as a target.

In addition, the information processing device 1 is capable ofperforming display control of performing the transition of the outputtarget region 90, with respect to the image displayed on the displaydevice 6, along the visual field transition path determined in thevisual field transition path determination processing.

In addition, the information processing device 1 is capable of allowinginformation of the visual field transition path determined in the visualfield transition path determination processing to correspond to theimage data, and of allowing the storage device 3 to store theinformation in the storage medium.

In addition, the information processing device 1 is capable of addingthe information of the visual field transition path determined in thevisual field transition path determination processing to the image data(for example, adding the information as meta data), and of transmittingthe information to the external device through the communication device4.

A hardware configuration example of the information processing device 1is illustrated in FIG. 4. Here, an example as a computer device isillustrated.

A CPU 51 of the information processing device 1 executes variousprocessings according to a program stored in a ROM 52 or a programloaded on a RAM 53 from a storage unit 59. In addition, data or the likenecessary for the CPU 51 to execute various processings is also suitablystored in the RAM 53.

The CPU 51, the ROM 52, and the RAM 53 are connected to each otherthrough a bus 54. In addition, an input/output interface 55 is alsoconnected to the bus 54.

A display 56 formed of a liquid crystal panel, an organic EL panel, orthe like, an input unit 57 formed of a keyboard, a mouse, or the like, aspeaker 58, a storage unit 59 configured of an HDD or the like, acommunication unit 60 are connected to the input/output interface 55.

In a case where the display device 6 of FIG. 3 is a device integratedwith the information processing device 1, the display 56 indicates thedisplay device 6. In a case where the display device 6 is a deviceseparated from the information processing device 1, the display 56 is adisplay as a computer device, and performs various information displaysfor a user interface with respect to an operator.

In a case where the manipulation device 5 of FIG. 3 is a deviceintegrated with the information processing device 1, the input unit 57indicates the manipulation device 5. In a case where the manipulationdevice 5 is a device separated from the information processing device 1,the input unit 57 indicates an input device used by the operator in thecomputer device.

The communication unit 60 performs communication processing through anetwork including the Internet, or communication with respect to thedevice of each of the peripheral units of FIG. 3.

In addition, as necessary, a drive 61 is connected to the input/outputinterface 55, a memory card 62 is mounted on the input/output interface55, and a computer program read out from the memory card 62, asnecessary, is installed in the storage unit 59, or data processed in theCPU 51 is stored in the input/output interface 55. It is obvious thatthe drive 61 may be a recording/reproducing drive with respect to aremovable storage medium such as a magnetic disk, an optical disk, and amagnetooptical disk.

In a case of such a computer device, processing as the informationprocessing device of the embodiment, that is, the visual fieldtransition path determination processing or the like is realized bysoftware activated by the CPU 51. A program configuring the software isdownloaded from the network, and is read out from the removable storagemedium, and thus, is installed in the computer device of FIG. 4.Alternatively, the program may be stored in advance in an HDD or thelike as the storage unit 59.

Then, the program is activated in the CPU 51, and thus, it is possibleto perform various processings described below.

Note that, the information processing device 1 of the embodiment is notlimited to a configuration including a single computer device asillustrated in FIG. 4, but may be configured by systemizing a pluralityof computer devices. The plurality of computer devices may be systemizedby a LAN or the like, or may be disposed in a remote location by a VPNor the like using the Internet. A plurality of information processingdevices may include an information processing device such as a computerdevice usable by a cloud computing service.

FIG. 5A and FIG. 5B illustrate a functional configuration example of theinformation processing device 1 of the embodiment, which is formed ofone or a plurality of computer devices or the like.

Note that, each function illustrated in FIG. 5A and FIG. 5B is afunction which is realized by processing executed by the CPU 51according to a program, in the information processing device 1. Here,processing of all or a part of each configuration described below may berealized by hardware.

In addition, in a case where each of the functions is realized bysoftware, it is not necessary that each of the functions is realized byan independent program. Processing of a plurality of functions may beexecuted by one program, or one function may be realized by cooperationof a plurality of program modules.

In addition, the respective functions may be dispersed in the pluralityof computer devices. Further, one of the functions may be realized bythe plurality of computer devices. Each of the functions may be realizedby the plurality of computer devices configured of a separate housing.

As illustrated in FIG. 5A, the information processing device 1 includesan image data acquisition unit 10, an object detection unit 11, a pathgeneration unit 12, a cut-out frame generation unit 13, a display datageneration unit 14, an input information acquisition unit 15, a displaycontrol unit 16, a storage control unit 17, and a communication controlunit 18.

The image data acquisition unit 10 acquires the image data (the movingimage data or the still image data) as the image having a continuousvisual field of 360 degrees in at least one direction. That is, forexample, the image data such as the celestial sphere image supplied fromthe image pickup device 2, the storage device 3, or the communicationdevice 4, is acquired.

For example, it is assumed that the image data acquisition unit 10 is areceiving device, an interface device, and an input buffering device,receiving the image data transmitted from the image pickup device 2, areceiving device, an interface device, and an input buffering device,receiving the image data reproduced in the storage device 3, or aninterface device or an input buffering device, inputting the image datareceived by the communication device 4.

The object detection unit 11 performs image analysis with respect to theimage data acquired by the image data acquisition unit 10, and detects aspecific object to be the subject. For example, a person, an animal,other movable bodies, person's face detection, a specific constructionproduct or object, and the like are detected in the image data.

Such object detection is performed in order to set a cut-out region tobe the output target region 90. This is because, for example, a regionin which the person's face is projected, is the output target region 90.

The image data and the information of the subject detected by the objectdetection unit 11 are used in the processing of the path generation unit12, the cut-out frame generation unit 13, and the display datageneration unit 14, as the designated information of the cut-out frame(the output target region 90).

Note that, the designated information of the cut-out frame is notlimited to be acquired from the object detection unit 11, but may bemanually labelled meta information, maker information applied to theimage data at the time of capturing, recognition information acquired byusing the other image processing such as face detection or persondetection.

As illustrated in FIG. 2B, the path generation unit 12 performs thevisual field transition path determination processing at the time of thetransition of the output target region 90 from a region of a certainsubject 100 to a region of the other subject 101.

The path generation unit 12 generates the visual field transition pathfrom the designated information a plurality of cut-out frames (outputtarget regions 90) by using the properties of the celestial sphereimage.

At this time, the path generation unit 12 determines the visual fieldtransition path by determining a visual field transition direction onthe basis of a position relationship between the output target region 90of a transition source and the output target region 90 of a transitiondestination, a movement direction, a movement speed, information ofreliability in a designated region, and the like.

For this reason, as illustrated in FIG. 5B, the path generation unit 12includes a specification unit 21 and a visual field transition pathdetermination unit 22.

The specification unit performs processing of specifying the outputtarget region 90 of the transition source and the output target region90 of the transition destination, with respect to the output targetregion 90 which is a partial region of the entire image HP, as theacquired image data.

The visual field transition path determination unit 22 performsprocessing of determining the visual field transition path from theoutput target region of the transition source to the output targetregion of the transition destination. For example, one path is selectedfrom the paths PT1 and PT2 illustrated in FIG. 2B, and the visual fieldtransition path is determined.

The processing of the path generation unit 12 will be described below indetail.

The cut-out frame generation unit 13 generates the information of theregion in the entire image HP, which is cut out as the output targetregion 90.

For example, a region including the specific subject detected by theobject detection unit 11 is set to the cut-out frame.

In addition, in a case of the transition from a certain output targetregion 90 to the other output target region 90, the information of eachcut-out frame to be the output target region 90 at each time point of atransition process is generated, according to the visual fieldtransition path determined in the path generation unit 12.

The display data generation unit 14 performs rendering processing foroutputting a cut-out image from the entire image HP, on the basis of theinformation of the cut-out frame generated by the cut-out framegeneration unit 13.

In addition, the display data generation unit 14 may perform processingfor superimposedly displaying various images (an icon, a thumbnail, andother images for a manipulation), a menu image to be separately display,and display data generation processing of the image for a manipulation,with respect to the cut-out image of the output target region 90.

In addition, the display data generation unit 14 may perform the displaydata generation processing including a transition setting image, anexistence presentation image, or the like, described below.

The input information acquisition unit 15 acquires the user manipulationinformation from the manipulation device 5 of FIG. 3 or the input unit57 of FIG. 4. For example, various manipulation information items of theuser, the designated information of the subject, and the like areacquired. The acquired information is used in the object detection unit11, the path generation unit 12, and the display data generation unit.

The display control unit 16 performs control of allowing the displaydata of the output target region 90 generated by the display datageneration unit to be displayed on the display device 6.

The storage control unit 17 performs control of allowing the displaydata generated by the display data generation unit, the information ofthe visual field transition path determined by the path generation unit12, and the like to be stored in the storage device 3.

The communication control unit 18 performs control of allowing thedisplay data generated by the display data generation unit, theinformation of the visual field transition path determined by the pathgeneration unit 12, and the like to be transmitted from thecommunication device 4 to the external device.

<3. Transition Processing of Output Target Region>

Transition processing of the output target region 90 according to theinformation processing device 1 described above, will be described.

Here, processing in a case where a plurality of different cut-outregions in the entire image HP are moved onto a spherical surface of acelestial sphere, will be described. For example, in FIG. 2A, a casewhere one path is determined as the visual field transition path in astate where the paths PT1 and PT2 are considered as illustrated in FIG.2B in order for the transition from the output target region 90 of thevisual line direction E1 to the output target region 90 of the visualline direction E2, will be exemplified.

First, an example of using the movement distance information in order todetermine the visual field transition path is described in FIG. 6.

FIG. 6A illustrates that the entire image HP of the celestial sphereimage is virtually in the shape of a rectangle. Note that, a right endand a left end of FIG. 6A are continuous. The same applies to FIG. 7A toFIG. 7F, FIG. 9A, FIG. 10A, and FIG. 11A described below.

In FIG. 6A, the output target region 90 including the subject 100 iscurrently displayed. At this time, it is assumed that the output targetregion 90 is transitioned such that the cut-out frame including thesubject 101 becomes the output target region 90.

Here, the transition of the output target region 90 will be described,prior to the determination of the transition path.

FIG. 7 illustrates an aspect of the transition of the output targetregion 90. The output target region 90 of FIG. 7A is an image regionincluding the subject 100. In the display device 6, the output targetregion 90 is displayed, and the user (the viewer) is in a state ofviewing the subject 100. From such a state, as illustrated in FIG. 7F,the cut-out region including the subject 101 becomes the output targetregion 90. That is, the user is in a state where the user is capable ofviewing the subject 101 in the display device 6.

For example, instantaneous switching from the state of FIG. 7A to thestate of FIG. 7F can also be performed, but in order to apply a displayeffect of the celestial sphere image to the user, and to apply a morenatural viewing environment within the celestial sphere image to theuser, the region to be displayed on the display device 6 may begradually moved within the entire image HP to reach the subject 101.Accordingly, the user is capable of feeling the transition of a scene tobe displayed, in such a manner as to look around.

For this reason, the output target region 90 is gradually transitionedfrom the state of FIG. 7A to FIG. 7B→FIG. 7C→FIG. 7D→FIG. 7E, andfinally reaches the state of FIG. 7F. Such a route is referred to as thevisual field transition path.

Note that, as illustrated in FIG. 7B to FIG. 7E, an example of how todetermine the output target region 90 even though the output targetregion 90 in the middle is set, in the transition process from theoutput target region 90 of the transition source to the output targetregion 90 of the transition destination, will be described.

FIG. 8 illustrates an interpolation model between two points on thespherical surface. The position of the subject 100 and the position ofthe subject 101 are set to two points, and interpolation is performedbetween two points.

The visual line direction Es is a visual line direction at a transitionstart time point with respect to the subject 100. The visual linedirection Ee is a visual line direction at a transition end time pointwith respect to the subject 101.

A plane obtained by cutting the sphere such that the position P0 and theposition of the subjects 100 and 101 are included in the plane, is setto an interpolation plane HH. The interpolation plane HH is a planedefined by the visual line direction at the transition start point andthe transition end point.

An orthogonal plane VH is a plane orthogonal to the interpolation planeHH. A direction vector of the orthogonal plane VH is set to a vector c.

In addition, a vector a of the visual line direction Es at thetransition start time point and a vector b of the visual line directionEe at the transition end time point are set.

The visual line direction to be interpolated, illustrated by a brokenline, is set to a vector d.

Note that, in the drawings and the following expressions, a vectorsymbol “→” is added to the vectors a, b, c, and d.

A control parameter of interpolating the visual line direction at thetransition start point and the transition end point is set to “t”.

t=0 and 1 is set. t=0 is a value at the transition start point, and t=1is a value at the transition end point.

First, an outer product of the vectors is used.

$\begin{matrix}{\overset{\rightarrow}{c} = \frac{\overset{\rightarrow}{a} \times \overset{\rightarrow}{b}}{{\overset{\rightarrow}{a} \times \overset{\rightarrow}{b}}}} & \left\lbrack {{Expression}\mspace{14mu} 1} \right\rbrack\end{matrix}$

Here, each of the vectors a and b is normalized by the vector of thevisual line direction Es at the transition start point and the visualline direction Ee at the transition end point.

In addition, an angle θ between the visual line direction Es at thetransition start point and the visual line direction Ee at thetransition end point is as follow.

θ=sin⁻¹ |{right arrow over (a)}×{right arrow over (b)}|  [Expression 2]

At this time, the visual line direction to be interpolated (the vectord) can be represented as described below, by using the control parametert.

{right arrow over (d)}(t)={right arrow over (a)} cos(θt)+({right arrowover (c)}×{right arrow over (a)})sin(θt)  [Expression 3]

Such a vector d is obtained by selecting each value in a range of 0 and1 with respect to the control parameter t, and thus, the cut-out regionto be interpolated can be determined as the output target region 90 ofthe transition process.

In a case of performing the visual field transition, it is consideredthat a transition time is set to be constant regardless of a distance tobe transitioned, or a transition speed is set to be constant. Further,it is also considered that the movement is performed slowly at thetransition start point and the transition end point, but the movement isperformed fast in the middle. Such control can be performed by selectingthe control parameter t.

In a case where an increase in the value of the control parameter t isset to a constant speed, with respect to the time of the visual field(the visual line direction) transition, the interpolation is performedat regular intervals, that is, the transition at a constant speed isrealized. The output target region 90 is moved at a constant speed, andthus, the user is capable of viewing a change in the image with anatural feeling.

In a case where the value of the control parameter t is set by dividingthe constant transition time into a predetermined number of times, thetransition time becomes constant, that is, the visual field transitionis performed such as in a case of transition of a shorter distance, themovement is performed slowly, and in a case of transition of a longerdistance, the movement is performed fast.

Further, in a case where the control parameter t gently increases nearthe visual line direction at the transition start point and thetransition end point (that is, near t=0 and near t=1 with respect to thecontrol parameter t), the transition also can be set such that thetransition start and end points are gently moved. In a case where theoutput target region 90 is gently moved near the start end of thetransition, it is considered that an image change is more comfortablefor the user.

It is obvious that it is also possible to accelerate or decelerate themovement speed by setting an increase in the control parameter t.

On the basis of the above description, returning to FIG. 6, adetermination method of the visual field transition path will bedescribed.

In the entire image HP of FIG. 6A, the illustrated paths PT1 and PT2 areassumed as a path from the output target region 90 including the subject100 to the output target region 90 including the subject 101, in whichthe output target region 90 is transitioned.

FIG. 6B illustrates an aspect in which the paths PT1 and PT2 are seenfrom an upper portion of a sphere. That is, FIG. 6B is a state in whichthe interpolation plane HH is seen from an upper portion on the vector cside of FIG. 8.

In this case, the path is determined such that the distance of theviewpoint movement from the visual line direction Es to the visual linedirection Ee becomes shorter on the spherical surface. That is, themovement distance information of the paths PT1 and PT2 is acquired, anda shorter path PT2 is selected as the visual field transition path.

Next, an example in which the visual field transition path is determinedby using the movement direction information of the subject of the outputtarget region 90 of the transition source, is illustrated in FIG. 9. Aswith FIG. 6A, in the entire image HP of FIG. 9A, the paths PT1 and PT2are assumed as the path in which the output target region 90 istransitioned from the output target region 90 including the subject 100to the output target region 90 including the subject 101. FIG. 9B is astate in which the interpolation plane HH is seen from an upper portion.

Here, the entire image HP is a moving image, and the subject 100 is amovable body other than the figure, for example, and is moved in anarrow M direction. That is, the position of the subject 100 is changedin the arrow M direction, according to the progress of the frame of themoving image.

Note that, in a case of a moving image, the output target region 90 ateach time of the transition process, which is selected by theinterpolation described in FIG. 7 and FIG. 8, is a region cut out from aframe corresponding to each time. That is, the user (the viewer) iscapable of viewing the display with a feeling that the visual linedirection is gradually moved on a moving image scene.

In this case, the visual field transition path is determined such thatthe direction thereof is identical to the direction of the motion in theoutput target region 90 of the transition source, for example, thedirection of the motion of the main subject 100 of the output targetregion 90.

That is, in this example, the path PT1 is the transition in the DR1direction, the path PT2 is the transition in the DR2 direction, and thesubject 100 is moved in the DR1 direction, and thus, the path PT1 isselected as the visual field transition path. Accordingly, the usermoves the visual line direction to an arrow EsM, and initially, thetransition is performed such that the scene is changed while followingthe subject 100 with eyes.

Note that, the information processing device 1 may analyze the directionof the motion of the subject existing in the output target region 90,for example, according to subject recognition and position comparison ofa plurality of frames, or may grasp the direction with reference to themeta data added to the image data. In addition, a global vector withrespect to the cut-out region may be analyzed, and may be a motiondirection of the main subject.

In addition, it is not necessary that the motion direction of thesubject 100 is limited to a horizontal direction with respect to theinterpolation plane HH. For example, as illustrated in FIG. 9C, eachcase of directions M1, M2, and M3 which are not horizontal to theinterpolation plane HH, is assumed. Note that, as illustrated in FIG. 8,the interpolation plane HH is a plane defined by the visual linedirection at the transition start point and the transition end point,and FIG. 9C is a diagram of a case in which it is considered thatcoordinates are rotated such that the transition source and thetransition destination are horizontal to each other.

The information processing device 1, for example, may determine whetherthe motion of the subject 100 is in the DR1 direction or the DR2direction, with reference to an orthogonal plane VH100 including theposition of the subject 100 at the transition start time point. That is,it may be discriminated whether to include a vector component of the DR1direction or to include a vector component of the DR2 direction.Accordingly, in any case of the illustrated directions M1, M2, and M3,the motion direction of the subject 100 is set to the DR1 direction.

In other words, it may be discriminated to which direction of thecandidate path (DR1 and DR2) a vector component is common, with respectto the directions M1, M2, and M3, and the like.

Next, an example in which the visual field transition path is determinedby using the movement direction information of the subject of the outputtarget region 90 of the transition destination, is described in FIG. 10.As with FIG. 6A, in the entire image HP of FIG. 10A, the paths PT1 andPT2 are assumed as the path in which the output target region 90 istransitioned from the output target region 90 including the subject 100to the output target region 90 including the subject 101. FIG. 10B is astate in which the interpolation plane HH is seen from an upper portion.

In this example, the entire image HP is a moving image, and the subject101 is a movable body other than the figure, and for example, is movedin the arrow M direction. That is, the position of the subject 101 ischanged in the arrow M direction, according to the progress of the frameof the moving image.

The visual field transition path is determined such that the directionthereof is identical to the direction of the motion in the output targetregion 90 of the transition destination, for example, the direction ofthe motion of the main subject 101 in the output target region 90 of thetransition destination.

In this example, the path PT1 is the transition of the DR1 direction,the path PT2 is the transition of the DR2 direction, and the subject 101is moved in the DR1 direction, and thus, the path PT1 is selected as thevisual field transition path. Accordingly, the transition is performedsuch that the user gradually moves the visual field from the visual linedirection Es, and then, finally follows the subject 101 with eyes toreach the output target region 90 of the subject 101.

In this case, in the information processing device 1, it is notnecessary that the motion direction of the subject 101 is limited to thehorizontal direction with respect to the interpolation plane HH. Forexample, as illustrated in FIG. 10C, each case of the directions M1, M2,and M3 which are horizontal to the interpolation plane HH is assumed.However, the information processing device 1, for example, may determinewhether the motion of the subject 101 is in the DR1 direction or the DR2direction, according to which vector component of the DR1 direction andthe DR2 direction is included, with reference to an orthogonal planeVH101 including the position of the subject 101 at the transition starttime point. Accordingly, in any case of the illustrated directions M1,M2, and M3, the motion direction of the subject 101 is set to the DR1direction.

However, there is a case where both of the subject 100 included in theoutput target region 90 of the transition source and the subject 101included in the output target region 90 of the transition source aremoved.

In this case, in a case where the motion directions are different fromeach other, it is considered that the visual field transition path isdetermined according to the subject having a faster movement speed.

That is, the information processing device 1 acquires movement speedinformation Vs of the subject 100 of the output target region 90 of thetransition source, and acquires movement speed information Ve of thesubject 101 of the output target region 90 of the transitiondestination. Simply, in the continuous frames, for example, a frame atthe transition start point and the next frame, a difference in positionswithin the entire image HP of the subject 100 (the number of pixels or adistance corresponding to a positional difference) may be set to themovement speed information Vs, and similarly, a positional difference ofthe subject 101 may be set to the movement speed information Ve.

It is obvious that the movement speed information items Vs and Ve may beset by calculating the actual speed value.

Then, the movement speed information items Vs and Ve are compared toeach other, and faster one is determined. In a case where the subject100 is faster than the subject 101, as illustrated in FIG. 9, the visualfield transition path is selected in the paths PT1 and PT2, according tothe motion direction of the subject 100. In a case where the subject 101is faster than the subject 100, as illustrated in FIG. 10, the visualfield transition path is selected in the paths PT1 and PT2, according tothe motion direction of the subject 101.

Note that, in a case where the motion direction of the subject 100 isthe same direction, it is obvious that the visual field transition pathmay be determined according to the motion direction.

Next, an example in which subject position information indicating theposition of the subject 101 existing in the output target region 90 ofthe transition destination in a frame after a required transition timehas elapsed, is used in order to determine the visual field transitionpath, is described in FIG. 11.

As with FIG. 6A, in the entire image HP of FIG. 11A, the transition fromthe output target region 90 including the subject 100 to the outputtarget region 90 including the subject 101 is considered. FIG. 11B is astate in which the interpolation plane HH is seen from an upper portion.

Here, the subject 101 as the transition destination exists in a positionin the entire image HP, which is represented as a subject 101 c, at thetransition start time point. However, the subject 101 is moved in the Mdirection in the drawing, and exists in the position of the illustratedsubject 101 in consideration of the position at the transition end timepoint.

In this case, the information processing device 1 confirms the positionof the subject 101 at the transition end point, and assumes the pathsPT1 and PT2 to be a candidate, with reference to the subject 101. Then,in the paths PT1 and PT2, one having a shorter distance is selected asthe visual field transition path.

Note that, in consideration of the premise that the transition time isset to be constant, the position of the subject 101 in a frame after thenumber of frames corresponding to the transition time may be determinedas the position of the subject 101 at the transition end point.

In addition, for example, in a case where the movement speed is set tobe constant at the time of the transition, in consideration of themovement speed of the output target region 90 and the movement speed ofthe subject 101, a time T1 when the output target region 90 reaches thesubject 101 in a case of the path PT1, and a time T2 when the outputtarget region 90 reaches the subject 101 in a case of the path PT2, maybe calculated, and a path having a shorter time may be selected. Thepath to be selected as a result is a path having a shorter movementdistance, with respect to the position of the subject 101 at thetransition end point.

As described above, an example of determining the visual fieldtransition path has been described, but it is obvious that adetermination method other than the exemplified determination method isalso considered.

In the visual field transition path determination processing, it is alsoconsidered that one having higher reliability is selected in thesubjects of the transition source and the transition destination, and aviewpoint movement direction (that is, the transition direction) isdetermined according to the movement direction of the selected subject,on the basis of reliability information of the region which is cut outand designated.

In a case where the region which is cut out and designated, is acquiredin the image processing such as “object detection” or “face detection”,likelihood of a detection accuracy is used as a specific example of thereliability information. In a case where the region which is cut out anddesignated, is acquired by the user input such as “labeling” or “maker”,a priority is used as a reliability. For example, in the region which iscut out and designated, it is also considered that the priority can beset to be higher in the user input than in the object detection, or thepriority can also be input by the user in the labeling, and thus, asubject having a higher reliability is selected according to thepriority.

FIG. 12 illustrates an example of the transition of the output targetregion 90 within the entire image HP, as a moving image. In an upperportion of the drawing, frames HP(t0), HP(t1), HP(t2), and HP(t3) areillustrated as the entire image HP. Such frames are frames at timepoints t0, t1, t2, and t3.

In a lower portion of the drawing, images which are cut out from theframes HP(t0), HP(t1), HP(t2), and HP(t3) at each of the time points tobe output target regions 90(t 0), 90(t 1), 90(t 2), and 90(t 3), areillustrated. That is, the image is an image that the user actuallyviews.

At the time point t0, the output target region 90(t 1) including anautomobile as the subject 100 is displayed. The transition is performedfrom the subject 100 to the other automobile of the subject 101.

Note that, both of the automobiles of the subjects 100 and 101 are movedin a left direction on the paper. For this reason, the visual fieldtransition path is determined such that the visual line direction ismoved to a left side, and the transition of the output target region 90is performed.

For example, at the time point t1, the output target region 90(t 1) ismoved to the left side within the entire image HP. Since it is acontinuous image of 360 degrees, the cut-out region is a region from aleft end portion to a right end portion of the frame HP(t1) of theentire image HP. The output target region 90(t 1) is in a state ofincluding only a tip end of the automobile of the subject 100 of thetransition source.

At the time point t2, the output target region 90(t 2) is further movedto the left side within the entire image HP. The output target region90(t 2) is in a state of including a part of the automobile as thesubject 101.

At the time point t3, the output target region 90(t 3) is further movedto the left side within the entire image HP, and the output targetregion 90(t 3) is in a state of including the entire automobile as thesubject 101. That is, the output target region 90(t 3) is in a statewhere the transition is ended.

For example, the visual field transition path is determined inconsideration of the motion of the subjects 100 and 101, and thus, it ispossible to apply the visual line movement with a natural feeling to theuser.

Various processing examples of the information processing device 1 fordetermining the visual field transition path as described above, will bedescribed as path determination processings I to V. The followingprocessing examples is processing which is executed by the pathgeneration unit 12 of the information processing device 1, according tothe function of the specification unit 21 and the visual fieldtransition path determination unit 22 illustrated in FIG. 5B.

FIG. 13A illustrates the path determination processing I. This is anexample of determining the visual field transition path by using themovement distance information described in FIG. 6.

The path generation unit 12 specifies the transition source and thetransition destination in Step S100. For example, the output targetregion 90 of the transition source is set to the currently displayedoutput target region 90. The output target region 90 of the transitiondestination is set to a region including a specific subject detected bythe object detection unit 11.

Note that, even though it will be described below, there is a case wherethe output target region 90 of the transition source is specified by adesignation manipulation or the like of the user.

In Step S101, the path determination unit 22 specifies the paths PT1 andPT2 to be the candidate described above (hereinafter, the paths PT1 andPT2 will be referred to as a “candidate path”), as the path from theoutput target region 90 of the determined transition source to theoutput target region 90 of the transition destination.

In Step S102, the path generation unit 12 calculates a viewpointmovement distance D1 with respect to the candidate path PT1. Theviewpoint movement distance D1 is a movement distance in a case ofadopting the candidate path PT1.

In Step S103, the path generation unit 12 calculates a viewpointmovement distance D2 with respect to the candidate path PT2. Theviewpoint movement distance D2 is a movement distance in a case ofadopting the candidate path PT2.

In Step S104, the path generation unit 12 compares the viewpointmovement distances D1 and D2 to each other.

Then, in Step S105, the path generation unit 12 selects a path having ashorter viewpoint movement distance in the candidate paths PT1 and PT2,and the selected path is determined as the visual field transition path.

FIG. 13B illustrates path determination processing II. This is anexample of determining the visual field transition path by using themovement direction information of the subject 100 of the transitionsource described in FIG. 9.

Note that, in the following flowchart, the same step numbers are appliedto the processings described above, and the detailed description will beomitted.

In Step S100, the path generation unit 12 specifies the transitionsource and the transition destination, and in Step S101, the pathgeneration unit 12 specifies the candidate paths PT1 and PT2.

Here, in Step S110, the path generation unit 12 allows the processing tobranch according to whether or not there is a motion with respect to thesubject 100 of the output target region 90 of the transition source.

In a case where there is no motion with respect to the subject 100, thesame processing as that of FIG. 13A is performed, that is, a path havinga shorter viewpoint movement distance is selected in the candidate pathsPT1 and PT2, and the selected path is determined as the visual fieldtransition path, according to Steps S102, S103, S104, and S105.

On this other hand, in a case where there is a motion with respect tothe subject 100 of the output target region 90 of the transition source,the path generation unit 12 proceeds to Steps S110 and Sill, anddiscriminates the movement direction of the subject 100 of thetransition source. That is, the DR1 direction or the DR2 direction inFIG. 9 is discriminated.

Then, in Step S112, the path generation unit 12 selects one of thecandidate paths PT1 and PT2 according to the discriminated motiondirection, and the selected path is determined as the visual fieldtransition path.

For example, in a case where the candidate path PT1 is a path on whichthe viewpoint movement of the DR1 direction is performed, the candidatepath PT2 is a path on which the viewpoint movement of the DR2 directionis performed, and the movement direction of the subject 100 is the DR1direction, the candidate path PT1 is selected.

FIG. 14A illustrates path determination processing III. This is anexample of determining the visual field transition path by using themovement direction information of the subject 101 of the transitiondestination described in FIG. 10.

In Step S100, the path generation unit 12 specifies the transitionsource and the transition destination, and in Step S101, the pathgeneration unit 12 specifies the candidate paths PT1 and PT2.

Here, in Step S120, the path generation unit 12 allows the processing tobranch according to whether or not there is a motion with respect to thesubject 101 of the output target region 90 of the transitiondestination.

In a case where there is no motion with respect to the subject 101, thesame processing as that of FIG. 13A is performed, that is, a path havinga shorter viewpoint movement distance is selected in the candidate pathsPT1 and PT2, and the selected path is determined as the visual fieldtransition path, according to Steps S102, S103, S104, and S105.

On this other hand, in a case where there is a motion with respect tothe subject 101 of the output target region 90 of the transitiondestination, the path generation unit 12 proceeds to Step S121 from StepS120, and the movement direction of the subject 101 of the transitiondestination is discriminated. That is, the DR1 direction or the DR2direction in FIG. 10 is discriminated.

Then, in Step S122, the path generation unit 12 selects one of thecandidate paths PT1 and PT2 according to the discriminated motiondirection, and determines the selected path as the visual fieldtransition path.

For example, in a case where the candidate path PT1 is a path on whichthe viewpoint movement of the DR1 direction is performed, the candidatepath PT2 is a path on which the viewpoint movement of the DR2 directionis performed, and the movement direction of the subject 101 is the DR1direction, the candidate path PT1 is selected.

FIG. 14B illustrates path determination processing IV. This is anexample of determining the visual field transition path by using thesubject position information at the transition end time point of thesubject 101 of the transition destination described in FIG. 11.

In Step S100, the path generation unit 12 specifies the transitionsource and the transition destination, and subsequently, in Step S120,the path generation unit 12 determines whether or not there is a motionwith respect to the subject 101 of the transition destination.

In a case where there is a motion, in Step S125, a position within theentire image HP of the subject 101 of the transition destination after atime Tm is determined, and then, the path generation unit 12 proceeds toStep S101. The time Tm is time required for the transition.Specifically, the path generation unit 12 determines the position of thesubject 101 in the corresponding frame after the time Tm from thecurrent frame.

In a case where there is no motion in the subject 101, the pathgeneration unit 12 proceeds to Step S101 without performing Step S125.

In Step S101, the path generation unit 12 specifies the candidate pathsPT1 and PT2. Here, in this case, in a case where there is a motion inthe subject 101, the transition destination is changed to the cut-outregion including the subject 101 after the time Tm, and then, thecandidate paths PT1 and PT2 are set.

Then, the path generation unit 12 performs the same processing as thatin FIG. 13A, that is, selects a path having a shorter viewpoint movementdistance in the candidate paths PT1 and PT2, and determines the selectedpath as the visual field transition path, according to Steps S102, S103,S104, and S105.

Accordingly, in a case where there is a motion in the subject 101 of thetransition destination, a path having a shorter transition distance isselected, with reference to the position of the subject 101 after thetime required for the transition.

FIG. 15 illustrates path determination processing V. This is an exampleof determining the visual field transition path by using the movementspeed information, in a case where both of the subject 100 of thetransition source and the subject 101 of the transition destination area moving subject.

The path generation unit 12 temporarily selects the visual fieldtransition path as Step S151. For example, the path generation unit 12executes the path determination processing I (S100 to S105) described inFIG. 13A, and performs temporary selection of temporarily setting thepath selected in the processing, that is, the path having a shorterviewpoint movement distance, to the visual field transition path.

In Step S152, the path generation unit 12 determines whether or not amovement direction component of the subject 100 of the transition sourceis common to the temporarily selected path. That is, it is discriminatedwhich component of the DR1 direction or the DR2 direction illustrated inFIG. 9 and FIG. 10 the movement direction of the subject 100 includes,and it is confirmed whether or not the movement direction component iscommon to the movement direction of the temporarily selected path.

In a case where the movement direction component of the subject 100 ofthe transition source is common to the movement direction of thetemporarily selected path, the path generation unit 12 proceeds to StepS153, and in this time, it is confirmed whether or not the movementdirection component (DR1 Direction/DR2 Direction) of the subject 101 ofthe transition destination is common to the temporarily selected path.

In a case where the movement direction component is common to thetemporarily selected path, the movement direction components of both ofthe subject 100 of the transition source and the subject 101 of thetransition destination are in common, and are also common to themovement direction of the temporarily selected path. Accordingly, inthis case, the path generation unit 12 proceeds to Step S156, anddetermines the temporarily selected path as the visual field transitionpath.

In Step S153, in a case where the movement direction component of thesubject 101 of the transition destination is not common to thetemporarily selected path, the motion directions of the subject 100 andthe subject 101 (the direction components of DR1 Direction/DR2Direction) are not coincident with each other.

In this case, in Step S154, the movement speed information Vs of thesubject 100 of the transition source and the movement speed informationVe of the subject 101 of the transition destination are calculated. Forexample, the actual movement speed is calculated, and then, is set tothe movement speed information items Vs and Ve. Then, in Step S155, itis determined whether or not Vs Ve is satisfied.

In a case where Vs Ve is satisfied, that is, the subject 100 of thetransition source is faster than the subject 101 of the transitiondestination, the path is selected according to the movement direction ofthe subject 100. In this case, the movement direction component of thesubject 100 is common to the temporarily selected path, and thus, thepath generation unit 12 proceeds to Step S156, and determines thetemporarily selected path as the visual field transition path.

On this other hand, in a case where Vs Ve is not satisfied, that is, thesubject 101 of the transition destination is faster than the subject 100of the transition source, the path is selected according to the movementdirection of the subject 101. In this case, the movement directioncomponent of the subject 101 is not common to the temporarily selectedpath. Therefore, the path generation unit 12 proceeds to Step S157, anddetermines a path which is not selected to the temporarily selected pathin Step S151, as the visual field transition path.

In Step S152, in a case where the movement direction component of thesubject 100 of the transition source is not common to the movementdirection of the temporarily selected path, the path generation unit 12proceeds to Step S158, and in this case, it is confirmed whether or notthe movement direction component of the subject 101 of the transitiondestination (DR1 Direction/DR2 Direction) is common to the temporarilyselected path.

In a case where the movement direction component is common to thetemporarily selected path, the movement direction components of both ofthe subject 100 of the transition source and the subject 101 of thetransition destination are not coincident with the movement direction ofthe temporarily selected path.

Accordingly, in this case, the path generation unit 12 proceeds to StepS161, and a path which is not selected to the temporarily selected pathin Step S151 is determined as the visual field transition path.

In Step S158, in a case where the movement direction component of thesubject 101 of the transition destination is common to the temporarilyselected path, the motion directions of the subject 100 and the subject101 (the direction components of DR1 Direction/DR2 Direction) are notcoincident with each other.

In this case, in Step S159, the movement speed information Vs of thesubject 100 of the transition source and the movement speed informationVe of the subject 101 of the transition destination are calculated.

Then, in Step S160, it is determined whether or not Vs≥Ve is satisfied.

In a case where Vs≥Ve is satisfied, that is, the subject 100 of thetransition source is faster than the subject 101 of the transitiondestination, the path is selected according to the movement direction ofthe subject 100. In this case, the movement direction component of thesubject 100 is not common to the temporarily selected path, and thus,the path generation unit 12 proceeds to Step S161, and determines a pathwhich is not selected to the temporarily selected path in Step S151, asthe visual field transition path.

On this other hand, in a case where Vs≥Ve is not satisfied, that is, thesubject 101 of the transition destination is faster than the subject 100of the transition source, the path is selected according to the movementdirection of the subject 101. In this case, the movement directioncomponent of the subject 101 is common to the temporarily selected path,and thus, the path generation unit 12 proceeds to Step S162, anddetermines the temporarily selected path as the visual field transitionpath.

Note that, in the processing of Steps S155 and S160, the determinationof Vs≥Ve is performed, but determination of Vs>Ve may be performed.

In addition, in a case where the movement speed information items Vs andVe are set to a value of a movement distance between predeterminedframes of the subjects 100 and 101, but not the actual speed value, inSteps S155 and S160, Vs≤Ve or Vs<Ve is determined.

In the processing example of FIG. 15 described above, in a case whereboth of the subjects 100 and 101 are moved, and the movement directioncomponents of both of the subjects 100 and 101 are different from eachother, the visual field transition path is determined by using themovement speed information.

Various path determination processings have been described so far, but aprocessing example other than the described example can be considered.It is obvious that a combination of the path determination processings Ito V is also assumed to be used.

That is, it is also considered that the path generation unit 12specifies the transition source and the transition destination, andthen, switches the processing such that in a case where there is nomotion in both of the subjects 100 and 101, the processing of Steps S101to S105 of FIG. 13A is performed, in a case where the subject 100 of thetransition source is moved, the processing of Steps S111 and S112 ofFIG. 13B is performed, in a case where the subject 101 of the transitiondestination is moved, the processing of Steps S121 and S122 of FIG. 14Aor the processing of FIG. 14B is performed, and in a case where both ofthe subjects 100 and 101 are moved, the processing of FIG. 15 isperformed.

4. Processing Using Transition Setting Image

Subsequently, an example will be described in which the informationprocessing device 1 displays a transition setting image representing aplurality of subject regions indicating an existence position of each ofa plurality of subjects within the moving image (an existence positionwhich is temporal and is visual within an image of 360 degrees), in alist, and the output target region 90 of the transition source and theoutput target region 90 of the transition destination are specified onthe basis of a selective instruction of the user with respect to thetransition setting image.

That is, a visual field transition path determination method using thelist display will be described.

The information processing device 1, for example, is capable ofgenerating the transition setting image as illustrated in FIG. 16A bytag information or detection information of the subject reflected on thecelestial sphere image, and of displaying the generated image on thedisplay device 6. In the transition setting image, a horizontal axis istime when the subject is reflected, and a vertical axis is a positionwhere the subject is reflected. 0 degrees and 360 degrees of thevertical axis are the same position as a visual field within the imageof 360 degrees.

The time of the horizontal axis corresponds to the progress of the frameof the moving image. The position of the vertical axis, for example, isa position represented in a range of 360 degrees in the horizontaldirection within the entire image HP.

The transition setting image, for example, represents the existenceposition of the specific subjects 100, 101, and 102 such as a figure,which are detected by the object detection unit 11. That is, in each ofthe subjects 100, 101, and 102, the existence position as a position ona time axis and in a range of 360 degrees is perspectively represented,and is displayed in a list. Each of the subjects 100, 101, and 102, forexample, is a thumbnail image of the subject. In a case of a figure, athumbnail image as a face image is considered.

In a case of the example of FIG. 16A, it is illustrated that the subject100 is in the same position without being moved almost even after timehas elapsed.

The subject 101 starts to move after the time point t2.

The subject 102 appears in the entire image HP at the time point t3, andafter that, slightly moves the position.

The information processing device 1, for example, such a transitionsetting image is generated by the display data generation unit 14, andis displayed on the display device 6.

The display data generation unit 14 is capable of generating thetransition setting image, on the basis of the detection information ofeach of the subjects 100, 101, and 102 from the object detection unit11.

The user is capable of arbitrarily designating a viewpoint movement withrespect to the display of the transition setting image.

An example is illustrated in FIG. 16B. For example, the user performs amanipulation of designating a thumbnail image with an oblique line, withrespect to each of the subjects 100, 101, and 102. For example, thereare the subject 100 at the time point t0, the subject 101 at the timepoint t3, and the subject 102 at a time point t6.

Accordingly, the information processing device 1 grasps a desiredviewpoint movement of the user. That is, the information processingdevice 1 grasps that in a case where the user instructs that first, aregion including the subject 100 is displayed as the output targetregion 90, the output target region 90 is transitioned to the positionof the subject 101 at the time point t3, and the output target region 90is transitioned to the position of the subject 102 at the time point t6,at the time of reproducing an image from the time point t0.

On the basis of this, the information processing device 1 (the pathgeneration unit 12) sets the transition of the output target region 90according to the time axis as illustrated by a solid arrow FIG. 16B.

Then, the information processing device 1 sets a cut-out range for eachframe according to the setting at the time of reproducing an image, andperforms display output with respect to the cut-out range as the outputtarget region 90.

Note that, a time interval of displaying the thumbnail image withrespect to each of the subjects 100, 101, and 102 is different accordingto a reproducing time length of the image contents, the size of thetransition setting image, the size of the thumbnail, and the like. Forexample, the interval of the time points t1, t2 . . . is not limited tocorrespond to a transition time length at the time of the visual fieldtransition. Accordingly, for example, a time when the visual fieldtransition from the subject 100 to the subject 101 at the time point t3is started, may be the time point t2, or may be immediately before thetime point t3, which is a time point after the time point t2.

The user is capable of setting the transition of the output targetregion 90 while grasping the position of the subject reflected on thecelestial sphere image and the time by seeing such a transition settingimage.

According to such designation of the user, the transition from a certainsubject to the other subject is set at a certain time point. In suchtransition, the path generation unit 12 performs the visual fieldtransition path determination processing.

An example is illustrated in FIG. 17. In FIG. 17, a case is illustratedin which the transition from the output target region 90 including thesubject 100 to the output target region 90 including the subject 102 isdesignated, at the time point t3.

At this time, the candidate paths PT1 and PT2 are assumed as the visualfield transition path. The candidate paths PT1 and PT2, for example,correspond to the paths PT1 and PT2 illustrated in FIG. 6, FIG. 9, FIG.10, and the like.

In a case of FIG. 17, it is detected that the subject 100 of thetransition source is not moved, but the subject 102 of the transitiondestination is moved. For example, in a case where the visual fieldtransition path is determined by the method described in FIG. 10 (thepath determination processing III of FIG. 14A), the subject 102 is movedin the DR2 direction, and thus, the path PT2 on which the transition ofthe DR2 direction is performed, is set to the visual field transitionpath.

FIG. 18A illustrates a case in which the transition from the outputtarget region 90 including the subject 101 to the output target region90 including the subject 102 is designated at the time point t3.

At this time, the candidate paths PT1 and PT2 are assumed as the visualfield transition path.

In a case of FIG. 18A, it is detected that both of the subject 100 ofthe transition source and the subject 102 of the transition destinationare moved. For example, in a case where the visual field transition pathis determined by the path determination processing V of FIG. 15, both ofthe subjects 101 and 102 are moved in the DR2 direction, and thus, thepath PT2 on which the transition of the DR2 direction is performed, isset to the visual field transition path.

FIG. 18B illustrates a case in which the transition from the outputtarget region 90 including the subject 101 to the output target region90 including the subject 102 is designated at a time point t5.

At this time, the candidate paths PT1 and PT2 are assumed as the visualfield transition path.

In a case of FIG. 18B, it is detected that the subject 100 of thetransition source is not moved, but the subject 102 of the transitiondestination is moved.

For example, the visual field transition path is determined by the pathdetermination processing III of FIG. 14A. In this case, the subject 102is moved in the DR1 direction, and thus, the path PT1 on which thetransition of the DR1 direction is performed, is set to the visual fieldtransition path.

Alternatively, in a case of performing processing considering theposition of the subject 102 at the time point t5 by using the pathdetermination processing IV, the path PT2 having a shorter movementdistance is set to the visual field transition path.

A processing example of the information processing device 1 using such atransition setting image is illustrated in FIG. 19. The processing ofFIG. 19 is processing to be executed by the function of each unit in theinformation processing device 1 (mainly, the image data acquisition unit10, the object detection unit 11, the path generation unit 12, thecut-out frame generation unit 13, the display data generation unit 14,the input information acquisition unit 15, and the display control unit16).

In Step S201, the information processing device 1 generates the displaydata as the transition setting image. That is, the display datageneration unit 14 generates the display data of the transition settingimage by using the information of the time and the position with respectto the specific subjects 100, 101, and 102 or the like, detected by theobject detection unit 11. Then, in Step S202, the information processingdevice 1 (the display control unit 16) displays the transition settingimage on the display device 6.

During the display of the transition setting image, the informationprocessing device 1 stands by designation input of the user in StepS203, and stands by display end of the transition setting image in StepS204.

The information processing device 1 detects the designation input of theuser, for example, the designation manipulation of the thumbnail asillustrated in FIG. 16B by the input information acquisition unit 15,and then, stores the designated information in Step S205.

In addition, the information processing device 1 proceeds to Step S206from Step S204, at a timing of the user manipulation or the display endof the transition setting image after a predetermined time has elapsed.

In Step S206, the information processing device 1 sets each transitionevent on the basis of the stored designated information. For example, ina case of the example of FIG. 16B, the transition from the subject 100to the subject 101 at the time point t3, and the transition from thesubject 101 to the subject 102 at the time point t6 are respectively setto the transition event.

In Step S207, one transition event is selected, and is set to aprocessing target. Then, the visual field transition path in thetransition event is determined by the path determination processing ofStep S208. That is, in Step S208, processing such as the path selectionprocessings I to V described above, is performed, and the visual fieldtransition path is determined.

In Step S209, it is confirmed whether or not the determination of thevisual field transition path with respect to all of the transitionevents is ended, and in a case where the determination is ended, in StepS207, an unprocessed transition event is selected, and similarly, thevisual field transition path is determined.

The determination of the visual field transition path with respect toall of the transition events is ended, and then, the informationprocessing device 1 proceeds to Step S210 from Step S209, and stores thevisual field transition of each of the transition events.

Note that, the storage in Step S205 and the storage in Step S210 may beperformed in the storage device 3 by the storage control unit 17, or maybe performed in the RAM 53 and the storage unit 59 of FIG. 4.

In addition, the information of the visual field transition path storedin Step S210, may be stored by being added to the image data as the metadata. Accordingly, it is possible for a reproducing device to recognizethe visual field transition path which is determined in advance withrespect to each of the transition events by the meta data, at the timeof reproducing, and to perform suitable transition display.

However, in a case where such transition can be designated, it is alsoassumed that a plurality of transition events occur in a comparativelyshort period. For example, there is a case where the transition iscontinuously performed in the manner of Subject 100→Subject 101→Subject102. In this case, in the process of each transition, the visual linedirection is interpolated, and thus, the progressive output targetregion 90 is cut out and is displayed.

It is considered that three or more points on the spherical surface (forexample, the subjects 100, 101, and 102, or the like) are used as atarget, and the points are interpolated.

In a case of performing the interpolation of three of more points, it isusually assumed that the interpolation between two points described inFIG. 8 described above is performed between two points.

FIG. 20A illustrates the subjects 100, 101, and 102 in a state where thespherical surface is opened to a plane. In this case, the interpolationbetween two points is performed between the subjects 100 and 101 andbetween the subjects 101 and 102. However, in this case, in a couplingportion surrounded by a broken line, an incontinuous visual linedirection is switched. At this time, there is a case where the user (theviewer) feels unnatural image transition.

Addition of a control point and spline interpolation are adopted as amethod of relaxing such an incontinuous direction switch.

First, as illustrated in FIG. 20B, a control point P is added onto ainterpolation curve obtained by the interpolation between two pointsdescribed above, at regular intervals. For example, the control point Pof dividing the interpolation curve into four is described.

Then, the position in the visual line direction as the subjects 100,101, and 102, and a spline curve completely passing through the controlpoint P are obtained. FIG. 20C illustrates a spline curve SC. It ispossible to perform two-dimensional interpolation in a coordinate systemwhere the celestial sphere image is opened to a plane, as a simpleexample.

According to such a spline curve SC, the visual line direction to beinterpolated is set, and the output target region 90 is cut out and isdisplayed according to each of the visual line directions, and thus, itis possible to perform the transition with a more natural feeling withrespect to the visual line movement.

5. Processing Using Existence Presentation Image

Subsequently, processing using the existence presentation image will bedescribed.

The information processing device 1 (the display data generation unit14) generates the display data displaying the existence presentationimage presenting a display candidate image existing in the entire imageHP, along with the image cut out as the output target region 90.

Then, an example will be described in which in a case where the inputinformation of generating the display candidate image is acquired, thepath generation unit 12 sets the output target region 90 being displayedto the transition source, and the information processing device 1specifies the transition destination on the basis of the inputinformation, and performs the visual field transition pathdetermination.

For example, a candidate region cut out as the output target region 90is presented to the user, and it is presented that the transition can beperformed to the region. Then, in a case where the user selects theregion, the transition is performed to the selected region.

FIG. 21A illustrates a display example including the existencepresentation image presenting the display candidate image.

The output target region 90 including a certain figure within the entireimage HP (a figure on a bike) is cut out and is displayed. In the framebeing displayed, the other display candidate (for example, the otherfigure) exists in the entire image HP, and a thumbnail image 80 of thedisplay candidate image is superimposedly displayed as the existencepresentation image.

The display candidate image, for example, is a specific subject detectedby the object detection unit 11, such as a figure. Alternatively, thedisplay candidate image may be a subject relevant to the designationmanipulation of the user, a subject represented by the meta data whichis added to the image data being reproduced, or the like.

The output target region 90 is a partial region of the entire image HP,and the user is not capable of seeing the other region within the entireimage HP, and thus, it is presented that the display candidate imageexists as the existence presentation image, to the user.

For example, as illustrated in FIG. 21A, in a case where the thumbnailimage 80 is set to the existence presentation image with respect to eachof the display candidate images, the user is capable of recognizing thecontents of the display candidate image (for example, the type offigure). In addition, in a case where the user wants to see the displaycandidate image, the thumbnail image 80 as the existence presentationimage is subjected to the designation manipulation, and thus, it ispossible to perform visual field transition of the display.

For example, according to a manipulation of selecting one of threethumbnail images 80 of FIG. 21A, the region of the subject as thedisplay candidate image relevant to the designated thumbnail image 80,is set to the transition destination, and the transition from the outputtarget region 90 of the current subject to the output target region 90of the subject of the transition destination is performed.

Note that, it has been described that the transition destination fromthe output target region 90 of the current subject is designated, but itis also considered that the transition source is designated by selectingthe thumbnail image 80. For example, the current subject is merely animage for designating the transition source or the transitiondestination. Then, it is also considered that both of the transitionsource and the transition destination are designated by the thumbnailimage 80, or only the transition source is designated (for example, thecurrent image is set to the transition destination). In a case of eachexample described below, it is considered that the transition source isdesignated.

FIG. 21B is an example of presenting the thumbnail image 80 as theexistence presentation image according to the time axis of the movingimage.

A track bar 81 is displayed on the display of the current output targetregion 90. In a case where the user touches the track bar 81 with amouse or the like, the thumbnail image 80 with respect to the subjectexisting in a frame of a time code corresponding to the position (thedisplay candidate image) is displayed.

A pointing method with respect to the track bar 81 is not limited to amethod of using the mouse, a tap manipulation with respect to a screen,or recognition of a hand position in virtual reality (VR) or the likemay be performed.

Then, a manipulation of selecting one thumbnail image 80 displayed asdescribed above is performed, and thus, the region of the subject as thedisplay candidate image relevant to the designated thumbnail image 80 isset to the transition destination, and the transition of the outputtarget region 90 from the region of the current subject is performed. Inthis case, the output target region 90 becomes the output target region90 of the frame at a time point when the designation is performed by thetrack bar 81.

In this case, the frame is switched in the time axis direction, and thetransition of the time axis direction and the positional transition maybe performed simultaneously or sequentially, or a frame in which thesubject as the designated display candidate image appears may besearched, and the transition may be temporally and positionallyperformed to the region of the frame.

FIG. 22A is an example of superimposedly displaying an arrow mark 83 asthe existence presentation image, on the display of the output targetregion 90. The arrow mark 83 presents that the subject as the displaycandidate image exists in an arrow direction of the celestial sphereimage.

As illustrated in FIG. 22B, in a case where the user performs amanipulation of designating a certain arrow mark 83 by a pointer 82 orthe like, for example, a manipulation such as mouseover, the informationprocessing device 1 displays the thumbnail image 80 of the subject asthe display candidate image existing in a direction illustrated by thearrow mark 83.

In such a state, in a case where the user performs the manipulation ofdesignating the thumbnail image 80, the information processing device 1sets the region of the subject as the display candidate image relevantto the designated thumbnail image 80, to the transition destination, andexecutes the transition of the output target region 90 from the regionof the current subject.

In this example, the thumbnail image 80 is not displayed at all times,and thus, the screen does not become cumbersome.

FIG. 23A is an example of presenting a radar image 70 as the existencepresentation image. The radar image 70, for example, is in the shape ofa circle of an image obtained by cutting a sphere, and a mark 71indicating the subject in the output target region 90 being currentlydisplayed is displayed in a vertically upward direction. That is, theposition of the figure on the bike is indicated within a range of 360degrees range.

In addition, the radar image 70 indicates the position of the subject asthe other display candidate image of the current frame by each of marks72, 73, and 74.

As illustrated in FIG. 23B, in a case where the user performs amanipulation of designating a certain mark 72 by the pointer 82 or thelike, the information processing device 1 displays the thumbnail image80 of the subject as the display candidate image indicated by the mark72.

In this state, in a case where the user performs the manipulation ofdesignating the thumbnail image 80, the information processing device 1sets the region of the subject as the display candidate image relevantto the designated thumbnail image 80, to the transition destination, andexecutes the transition of the output target region 90 from the regionof the current subject.

In this example, the thumbnail image 80 is not displayed at all times,and thus, the screen does not become cumbersome.

A processing example of the information processing device 1 relevant tothe display of the existence presentation image and the visual fieldtransition according to the user designation is illustrated in FIG. 24.The processing of FIG. 24 is processing executed by the informationprocessing device 1 according to each function of FIG. 5A.

In Step S301, the information processing device 1 starts the supply ofthe display data of the output target region 90 on which the existencepresentation image is superimposed, with respect to the display device6, and starts the display output in the display device 6. For example,display as illustrated in FIG. 22A or FIG. 23A is started.

In Steps S302, S303, S304, and S305, the information processing device 1monitors the user manipulation or the like according to the function ofthe input information acquisition unit 15.

For example, in a case where a thumbnail request manipulation such asthe designation of the arrow mark 83 of FIG. 22B or the designation ofthe mark 72 of FIG. 23B is detected, the information processing device 1proceeds to Step S310, and starts the display of the correspondingthumbnail image 80.

After that, in a case where the thumbnail request manipulation is ended,the information processing device 1 proceeds to Step S311 from StepS303, and ends the display of the thumbnail image 80. For example, asillustrated in FIG. 22B or FIG. 23B, in a state where the thumbnailimage 80 is displayed, there is a case where the user moves the positionof the pointer 82 to the other position without performing thedesignation manipulation of the thumbnail image 80.

During the display of the thumbnail image 80, in a case where themanipulation of designating the thumbnail image 80 is detected, theinformation processing device 1 recognizes the detected manipulation asthe designation of the visual field transition with respect to thesubject of the display candidate image corresponding to the thumbnailimage 80, and proceeds to Step S312 from Step S304.

In Step S312, for example, in any processing of the path determinationprocessings I to V described above, the visual field transition path forthe transition from the current output target region 90 to the outputtarget region 90 of the subject designated by the thumbnail image 80, isdetermined.

Then, in Step S313, the information processing device 1 sequentiallyoutputs the display data of the output target region 90 to which thecut-out region is moved, while performing the interpolation of thevisual field direction in the determined path, and thus, executes thevisual field transition in the display device 6. In Step S314, at a timepoint when it is determined that the transition is completed, thetransition processing is ended.

In a case where a display end timing is detected by the usermanipulation, reproducing end of the display contents as the celestialsphere image, or the like, the information processing device 1 proceedsto Step S315 from Step S305, ends the display output, and ends a set ofprocessings.

According to the processing described above, in a state where it isexplicit for the user that there is the other display candidate image,by the existence presentation image, it is possible to display theoutput target region 90 which is a part of the entire image HP.

Note that, in the processing example described above, cases of FIG. 22and FIG. 23 are assumed, but as illustrated in FIG. 21A, in a case wherethe thumbnail image 80 is display as the existence presentation imagefrom the beginning, Steps S302, S303, S310, and S311 of FIG. 24 are notnecessary.

In addition, as illustrated in FIG. 21B, in a case where the track bar81 is displayed, time code designation on the track bar 81 may beconsidered as the thumbnail request manipulation of Step S302, and theend of the time code designation on the track bar 81 may be consideredas thumbnail request end of Step S303.

6. Conclusion and Modification Example

In the embodiment described above, the following effects can beobtained.

The information processing device 1 of the embodiment includes thespecification unit 21 specifying the output target region 90 of thetransition source and the output target region 90 of the transitiondestination in the output target region 90 which is the partial regionof the entire image HP, which is an image having a continuous visualfield of 360 degrees in at least one direction, and the visual fieldtransition path determination unit 22 automatically determining thevisual field transition path from the output target region 90 of thetransition source to the output target region 90 of the transitiondestination.

In the transition of the output target region 90 imitating the viewpointmovement of the audience, in a case where the transition source and thetransition destination in the entire continuous image of 360 degrees areassumed, the visual field transition path from the output target regionof the transition source to the output target region of the transitiondestination as the continuous image can be variously considered.Therefore, the visual field transition path is automatically determined,and thus, visually smooth transition can be provided to the viewer, onthe display imitating the viewpoint movement as the transition from thetransition source to the transition destination.

In the information processing device 1 of the embodiment, the pathgeneration unit 12 (the visual field transition path determination unit22) determines one of the candidate path progressing to one directionfrom the output target region 90 of the transition source towards theoutput target region 90 of the transition destination within the entireimage HP (for example, the path PT1) and the candidate path progressingto the direction opposite to one direction (for example, the path PT2),as the visual field transition path.

In the transition of the output target region, in a case where thetransition source and the transition destination are assumed in theentire continuous image of 360 degrees, a plurality of pathstransitioned from the transition source to the transition destination asthe continuous image are assumed. In particular, in consideration of theposition relationship between the transition source and the transitiondestination in the entire continuous image of 360 degrees, it isconsidered that the transition from the transition source to thetransition destination (the viewpoint movement) is in the directionopposite to one direction on the circumference of 360 degrees of theentire image. One of the plurality of paths is set to be automaticallyselected. Therefore, in the candidate path progressing to one directionand the candidate path progressing to the direction opposite to onedirection, for example, a visually desired path is automaticallydetermined as the visual field transition path.

In particular, it is determined that in which direction transitionbecomes a more smooth viewpoint movement by comparing transition in onedirection to transition in a direction opposite to one direction in acircumference direction, and thus, it is possible to determine asuitable visual field transition path according to comparatively easyprocessing.

In the transition path determination processings I, II, III, and IV ofthe embodiment, for example, the movement distance information (D1 andD2) indicating a movement distance from the output target region of thetransition source to the output target region of the transitiondestination on the entire image for each of a plurality of candidatepaths, is used in order to determine the visual field transition path.

For example, a path having a shorter movement distance is selected, andthus, it is possible to efficiently present the viewpoint movement fromthe transition source to the transition destination. It is possible torealize smooth transition which is not visually redundant. For example,the movement distance information is used in order to determine thevisual field transition path as described above, and thus, there is acase where a comfortable feeling can be realized at the time of beingviewed by the user view.

In the transition path determination processings II and V of theembodiment, in the entire image HP of the moving image, the movementdirection information indicating the movement direction in the entireimage HP on the progress of the moving image, of the subject 100existing in the output target region 90 of the transition source, isused in order to determine the visual field transition path.

That is, in the output target region 90 of the transition source, in acase where a figure to be the subject 100 and the other movable body aremoved in a certain direction on the moving image, the movement directionis one factor for determining the visual field transition path.

The motion of the subject 100 of the transition source affects theviewpoint of the viewer.

Accordingly, in a case where the viewpoint is moved from the transitionsource to the transition destination, the visual field transition pathat the time of transition is determined in consideration of the movementdirection of the subject 100 of the transition source, which has beenseen so far, and thus, the viewer is capable of feeling a smoothviewpoint movement.

In the transition path determination processings III, IV, and V of theembodiment, the movement direction information indicating the movementdirection within the entire image HP on the progress of the movingimage, of the subject 101 existing in the output target region 90 of thetransition destination, is used in order to determine the visual fieldtransition path.

That is, in the output target region 90 of the transition destination,in a case where the figure to be the subject 101 and the other movablebody are moved in a certain direction on the moving image, the movementdirection is one factor for determining the visual field transitionpath.

The fact that the motion of the subject affects the viewpoint of theviewer, also applies to the subject 101 of the transition destination.In a case where the viewpoint is moved from the transition source to thetransition destination, the visual field transition path at the time oftransition is determined in consideration of the movement direction ofthe subject 101 of the transition destination, which appears accordingto the transition, and thus, the viewer is capable of feeling a smoothviewpoint movement.

In the transition path determination processing V of the embodiment, themovement speed information (Vs and Ve) indicating the movement speedwithin the entire image HP on the progress of the moving image, of thesubject 101 existing in the output target region 90 of the transitiondestination or the subject 100 existing in the output target region 90of the transition source, is used in order to determine the visual fieldtransition path.

That is, in a case where one or both of the images of each of thetransition source and the transition destination (the figure to be thesubject and the other movable body) are moved on the moving image, themovement speed is one factor for determining the visual field transitionpath.

The motion of the subjects 100 and 101 affects the viewpoint movement ofthe viewer, and in particular, the condition of an influence on theviewpoint is different according to a fast motion and a slow motion.Therefore, the visual field transition path at the time of transition isdetermined in consideration of the movement speed of the subjects 100and 101, and thus, it is possible to realize transition of allowing theviewer to feel a smooth viewpoint movement.

In the transition path determination processing IV of the embodiment,the subject position information indicating the position of the subject101 existing in the output target region 90 of the transitiondestination in the frame after the required transition time has elapsed,is used in order to determine the visual field transition path.

In continuous transition from the transition source to the transitiondestination (the viewpoint movement), a transition time is required. Ina case where there is a motion in the subject of the transitiondestination, it is assumed that the position of the subject within theentire image after the required transition time, is different from theposition in the frame at the transition start time point. Therefore, thevisual field transition path is determined in consideration of theposition of the subject of the transition destination when the requiredtransition time has elapsed.

Accordingly, it is possible to realize transition of allowing the viewerto feel a smooth viewpoint movement on the assumption of the motion ofthe subject 101 of the transition destination.

In the embodiment, the display or the transition setting of thetransition setting image described in FIG. 16 or the like can beperformed. That is, the information processing device 1 presents thetransition setting image representing the plurality of subject regionsindicating the existence position within the moving image of each of theplurality of subjects, which are the partial region included in themoving image, with respect to the entire image HP as the moving image,in a list, to the user. Then, the information processing device 1 (thepath generation unit 12) specifies the output target region 90 of thetransition source and the output target region 90 of the transitiondestination, on the basis of the selective instruction of the user(refer to FIG. 19).

That is, it is possible to display the positions of each of the subjectswithin the entire image at each time point in a list with respect to theuser who is the viewer of the image, and for example, to select thesubject. Accordingly, the movement of the output target region accordingto the progress of the moving image, that is, the viewpoint movement isperformed, and thus, it is possible to present the subject that the userwants to see, while performing the viewpoint movement.

In this case, the visual field transition path at the time of thetransition between the subjects (the viewpoint movement) is suitablydetermined, and thus, the user is capable of feeling a smooth andnatural viewpoint movement.

In addition, the user is capable of setting a viewing field angle by thetransition setting image, while perspectively viewing the entirecelestial sphere video, and thus, a viewing path considering theposition relationship of the subjects and the time is easily set.

In the embodiment, as illustrated in FIG. 21 to FIG. 24, the processingusing the existence presentation image is performed. That is, theinformation processing device 1 includes the display data generationunit 14 generating the display data displaying the image of the partialregion which is the output target region 90 within the entire image HP,and the existence presentation image (80, 81, 83, 70, and the like)presenting the display candidate image existing in the still image orthe moving image as the entire image HP. In addition, the informationprocessing device 1 includes the input information acquisition unit 15acquiring the input information of designating the display candidateimage, which is performed with respect to the display based on thedisplay data. Then, the path generation unit 12 (the specification unit21) specifies the output target region 90 of the transition destinationor the output target region 90 of the transition source, on the basis ofthe user manipulation of designating the display candidate image, whichis performed with respect to the display based on the display dataincluding the image of the partial region which is the output targetregion within the entire image, and the existence presentation imagepresenting the display candidate image existing in the still image orthe moving image as the entire image (Step S312 of FIG. 24).

That is, the existence of the subject of the other display candidate ispresented on the image being displayed as the existence presentationimage. Then, the user performs the manipulation of selecting a certaindisplay candidate image by the manipulation with respect to theexistence presentation image, or the like, and thus, the image displaysuch as the viewpoint movement to the region including the displaycandidate image, is realized.

In this case, the visual field transition path at the time of thetransition from the region being currently displayed to the region ofthe display candidate image (the viewpoint movement) is suitablydetermined, and thus, the user is capable of feeling a smooth andnatural viewpoint movement.

In addition, actually, the subject as the display candidate image hasvarious sizes. In the embodiment, the subject is presented as thethumbnail image 80 of the same size. Accordingly, it is possible toequivalently confirm the respective display candidates, and toaccelerate free selection of the user. In addition, it is possibleprevent the display candidate from being presented as a needlessly largeimage and the image of the current output target region 90 from beingdisturbed as much as possible.

In the embodiment, the thumbnail image 80 of the display candidate imageis set to the existence presentation image, and one or a plurality ofthumbnail images 80 is displayed along with the image of the currentoutput target region 90 (refer to FIG. 21 to FIG. 23).

That is, existence presentation is performed by a method ofsuperimposing the existing display candidate image, for example, on apart of the current output target region as the thumbnail image 80 of auniform size, or the like. Accordingly, the user is capable of easilyrecognizing that there are other notable images, or the image contents,and for example, also easily understands the manipulation for theviewpoint movement.

In the embodiment, an example is described in which the existencepresentation image is set to an image which is displayed in the positionaccording to the position relationship with respect to the currentoutput target region 90 of the display candidate image indicated by theexistence presentation image, on the display image of the current outputtarget region 90 (refer to FIG. 22 and FIG. 23).

For example, the existence presentation image with respect to thedisplay candidate image existing in the upper portion on the screen withrespect to the current output target region 90, is presented in theupper portion on the current screen as with the arrow mark 83 of FIG.22. Similarly, the existence presentation image with respect to thedisplay candidate image existing in the right portion on the screen withrespect to the current output target region 90, is presented in theright portion on the current screen as with the arrow mark 83.

Accordingly, the user is capable of easily recognize the positionrelationship of the display candidate image from the current image.Here, a case of an image of 360 degrees is described, and thus, forexample, the display candidate image to be in the right portion, can becontinuously transitioned from the left side. Therefore, it is suitablethat the position relationship indicates a closer one in the rightportion and the left portion.

In addition, the thumbnail image 80 is displayed according to thedesignation of the arrow mark 83, and thus, it is possible for the userto easily discriminate the subject existing in the correspondingposition, and to easily select desired visual field transition.

Further, the thumbnail image 80 is not displayed at all times, and thus,it is possible to provide a comfortable viewing environment withoutmaking the screen cumbersome.

In the embodiment, as illustrated in FIG. 21B, an example is describedin which the thumbnail image 80 as the existence presentation image isdisplayed corresponding to the time axis position of the moving imagecontents of the display candidate image (a position on the track bar 81)to which the existence presentation image corresponds.

In a case of the moving image contents, each existence state of thedisplay candidate image varies in the time axis direction. Therefore, itis possible to confirm the existence of the display candidate image onthe time axis.

Accordingly, the user easily recognizes the existence of each displaycandidate image on the time axis. Then, the reflected object is easilygrasped without being limited to the same time as that of the framebeing viewed, and variousness of the designation of the transitiondestination and easiness of the transition designation with respect tothe temporal direction can be applied to the user.

In the information processing device according to the present technologydescribed above, it is considered that the display data generation unitgenerates the display data including the existence presentation imageindicating the position relationship between the display candidate imageand the current output target region.

For example, an image indicating a relative position of the displaycandidate image with respect to the current output target region 90 bythe marks 71, 72, 73, and 74 on the radar image 70 of FIG. 23 is set tothe existence presentation image.

Accordingly, the user is capable of more easily and obviouslyrecognizing the position relationship of each of the display candidateimages. Accordingly, the transition imitating the viewpoint movement isalso felt with a more uncomfortable feeling.

In addition, the thumbnail image 80 is displayed according to thedesignation of the marks 72, 73, and 74, or the like, and thus, it ispossible for the user to easily discriminate the subject existing in theposition, and to easily select desired visual field transition.

Further, the thumbnail image 80 is not displayed at all times, and thus,it is possible to provide a comfortable viewing environment withoutmaking the screen cumbersome.

A program of the embodiment is a program of allowing the informationprocessing device to execute a specification procedure of specifying theoutput target region 90 of the transition source and the output targetregion 90 of the transition destination in the output target region 90which is the partial region of the entire image HP, which is the imagehaving a continuous visual field of 360 degrees in at least onedirection, and a visual field transition path determination procedure ofautomatically determining the visual field transition path from theoutput target region 90 of the transition source to the output targetregion 90 of the transition destination. For example, the program is aprogram of allowing an information processing device (a microcomputer, adigital signal processor (DSP), or the like) to execute the processingof FIG. 13A, FIG. 13B, FIG. 14A, FIG. 14B, FIG. 15, FIG. 19, FIG. 24,and the like is a program.

According to the program, an arithmetic processing device having afunction as the information processing device 1 of the embodiment can berealized.

Such a program can be recorded in various recording mediums. Inaddition, the program can be recorded in advance in an HDD as arecording medium which is embedded in a device such as a computerdevice, a ROM in a microcomputer including a CPU, and the like. Inaddition, the program can be temporarily or permanently recorded in aremovable recording medium such as a semiconductor memory, a memorycard, an optical disk, a magnetooptical disk, and a magnetic disk. Inaddition, such a removable recording medium can be provided as so-calledpackage software.

In addition, such a program can be installed in a personal computer orthe like from a removable recording medium, or can be downloaded from adownload site through a network such as a LAN and the Internet.

The present technology is not limited to the examples of the embodimentor the modification example or the like described in the embodiment, andmore various modification examples are assumed.

The information of the visual field transition path determined in thetransition path determination processing is used as the display controlof the time point, and it is also considered that the information isstored corresponding to the image data, or is added to the image data asthe meta data, and thus, can be used even at the time of reproducinglater.

In addition, it is also assumed that determination information of thevisual field transition path is transmitted to the external device.

In addition, in the path determination processing of the embodiment,selecting one path after calculating the candidate path is described asthe processing example, but the final visual field transition path maybe directly determined without calculating the candidate path. Forexample, in a case where the visual field transition path is determinedby using the motion direction of the subject, it is possible todetermine the final visual field transition path according to themovement direction of the subject, without performing processing ofobtaining a plurality of candidate paths.

As described above, it is not necessary that the information processingdevice 1 is configured of one device. A part of the function of theinformation processing device 1 of the embodiment may be executed in theother information processing device connected to the system, or may beexecuted in the external information processing device by so-calledcloud computing.

Note that, the effects described herein are merely an example, but thepresent technology is not limited thereto, and other effects may beobtained.

Note that, the present technology is also capable of adopting thefollowing configurations.

(1) An information processing device, including:

a specification unit that specifies a transition source output targetregion and a transition destination output target region, as an outputtarget region which is a partial region of an entire image, which is animage including a continuous visual field of 360 degrees in at least onedirection; and

a visual field transition path determination unit that automaticallydetermines a visual field transition path from the transition sourceoutput target region to the transition destination output target region.

(2) The information processing device according to (1) described above,in which

the visual field transition path determination unit determines one of acandidate path progressing to one direction from the transition sourceoutput target region towards the transition destination output targetregion, and a candidate path progressing to a direction opposite to theone direction, as the visual field transition path.

(3) The information processing device according to (1) or (2) describedabove, in which

the visual field transition path determination unit uses movementdistance information indicating a movement distance from the transitionsource output target region to the transition destination output targetregion for each of a plurality of candidate paths, in order to determinethe visual field transition path.

(4) The information processing device according to any one of (1) to (3)described above, in which

the entire image is a moving image, and

the visual field transition path determination unit uses movementdirection information indicating a movement direction in the entireimage of a subject existing in the transition source output targetregion on progress of the moving image, in order to determine the visualfield transition path.

(5) The information processing device according to any one of (1) to (4)described above, in which

the entire image is a moving image, and

the visual field transition path determination unit uses movementdirection information indicating a movement direction in the entireimage of a subject existing in the transition destination output targetregion on progress of the moving image, in order to determine the visualfield transition path.

(6) The information processing device according to any one of (1) to (5)described above, in which

the entire image is a moving image, and

the visual field transition path determination unit uses movement speedinformation indicating a movement speed in the entire image of a subjectexisting in the transition destination output target region or a subjectexisting in the transition source output target region on progress ofthe moving image, in order to determine the visual field transitionpath.

(7) The information processing device according to any one of (1) to (6)described above, in which

the entire image is a moving image, and

the visual field transition path determination unit uses subjectposition information indicating a position of a subject existing in thetransition destination output target region in a frame after a requiredtransition time has elapsed, in order to determine the visual fieldtransition path.

(8) The information processing device according to any one of (1) to (7)described above, in which

the entire image is a moving image, and

the specification unit specifies

the transition source output target region and the transitiondestination output target region, on a basis of a selective instructionof a user with respect to a transition setting image representing aplurality of subject regions each of which is a partial region includedin the moving image and indicates an existence position of each of aplurality of subjects in the moving image, in a list.

(9) The information processing device according to any one of (1) to (8)described above, in which

the specification unit specifies

the transition destination output target region or the transition sourceoutput target region, on a basis of a user manipulation of designating adisplay candidate image, that is performed with respect to a displaybased on display data including an image of a partial region to be theoutput target region in the entire image and an existence presentationimage representing a display candidate image existing in a still imageor a moving image as the entire image.

(10) The information processing device according to (9) described above,in which

the existence presentation image is a thumbnail image of the displaycandidate image, and

the display data includes an image of a current output target region anda thumbnail image of the display candidate image.

(11) The information processing device according to (9) or (10)described above, in which

the existence presentation image is provided at a position correspondingto a position relationship of the display candidate image indicated bythe existence presentation image with respect to the image of thepartial region.

(12) The information processing device according to any one of (9) to(11) described above, in which

the display data includes an existence presentation image correspondingto a display candidate image existing in a designated time axisposition, that corresponds to the time axis position designated in theimage of the partial region to be the output target region.

(13) The information processing device according to any one of (9) to(12) described above, in which

the display data includes an existence presentation image indicating aposition relationship between the display candidate image and the imageof the partial region to be the output target region.

(14) An information processing method executed by an informationprocessing device, the method including:

a specification step of specifying a transition source output targetregion and a transition destination output target region, in an outputtarget region which is a partial region of an entire image, which is animage including a continuous visual field of 360 degrees in at least onedirection; and

a visual field transition path determination step of automaticallydetermining a visual field transition path from the transition sourceoutput target region to the transition destination output target region.

(15) A program that causes an information processing device to execute:

a specification step of specifying a transition source output targetregion and a transition destination output target region, in an outputtarget region which is a partial region of the entire image, which is animage having a continuous visual field of 360 degrees in at least onedirection; and

a visual field transition path determination step of automaticallydetermining a visual field transition path from the transition sourceoutput target region to the transition destination output target region.

REFERENCE SIGNS LIST

-   -   1 information processing device    -   2 image pickup device    -   3 storage device    -   4 communication device    -   5 manipulation device    -   6 display device    -   10 image data acquisition unit    -   11 object detection unit    -   12 path generation unit    -   13 cut-out frame generation unit    -   14 display data generation unit    -   15 input information acquisition unit    -   16 display control unit    -   17 storage control unit    -   18 communication control unit    -   21 specification unit    -   22 visual field transition path determination unit    -   90 output target region    -   100,101 subject

1. An information processing device, comprising: a specification unitthat specifies a transition source output target region and a transitiondestination output target region, as an output target region which is apartial region of an entire image, which is an image including acontinuous visual field of 360 degrees in at least one direction; and avisual field transition path determination unit that automaticallydetermines a visual field transition path from the transition sourceoutput target region to the transition destination output target region.2. The information processing device according to claim 1, wherein thevisual field transition path determination unit determines one of acandidate path progressing to one direction from the transition sourceoutput target region towards the transition destination output targetregion, and a candidate path progressing to a direction opposite to theone direction, as the visual field transition path.
 3. The informationprocessing device according to claim 1, wherein the visual fieldtransition path determination unit uses movement distance informationindicating a movement distance from the transition source output targetregion to the transition destination output target region for each of aplurality of candidate paths, in order to determine the visual fieldtransition path.
 4. The information processing device according to claim1, wherein the entire image is a moving image, and the visual fieldtransition path determination unit uses movement direction informationindicating a movement direction in the entire image of a subjectexisting in the transition source output target region on progress ofthe moving image, in order to determine the visual field transitionpath.
 5. The information processing device according to claim 1, whereinthe entire image is a moving image, and the visual field transition pathdetermination unit uses movement direction information indicating amovement direction in the entire image of a subject existing in thetransition destination output target region on progress of the movingimage, in order to determine the visual field transition path.
 6. Theinformation processing device according to claim 1, wherein the entireimage is a moving image, and the visual field transition pathdetermination unit uses movement speed information indicating a movementspeed in the entire image of a subject existing in the transitiondestination output target region or a subject existing in the transitionsource output target region on progress of the moving image, in order todetermine the visual field transition path.
 7. The informationprocessing device according to claim 1, wherein the entire image is amoving image, and the visual field transition path determination unituses subject position information indicating a position of a subjectexisting in the transition destination output target region in a frameafter a required transition time has elapsed, in order to determine thevisual field transition path.
 8. The information processing deviceaccording to claim 1, wherein the entire image is a moving image, andthe specification unit specifies the transition source output targetregion and the transition destination output target region, on a basisof a selective instruction of a user with respect to a transitionsetting image representing a plurality of subject regions each of whichis a partial region included in the moving image and indicates anexistence position of each of a plurality of subjects in the movingimage, in a list.
 9. The information processing device according toclaim 1, wherein the specification unit specifies the transitiondestination output target region or the transition source output targetregion, on a basis of a user manipulation of designating a displaycandidate image, that is performed with respect to a display based ondisplay data including an image of a partial region to be the outputtarget region in the entire image and an existence presentation imagerepresenting a display candidate image existing in a still image or amoving image as the entire image.
 10. The information processing deviceaccording to claim 9, wherein the existence presentation image is athumbnail image of the display candidate image, and the display dataincludes an image of a current output target region and a thumbnailimage of the display candidate image.
 11. The information processingdevice according to claim 9, wherein the existence presentation image isprovided at a position corresponding to a position relationship of thedisplay candidate image indicated by the existence presentation imagewith respect to the image of the partial region.
 12. The informationprocessing device according to claim 9, wherein the display dataincludes an existence presentation image corresponding to a displaycandidate image existing in a designated time axis position, thatcorresponds to the time axis position designated in the image of thepartial region to be the output target region.
 13. The informationprocessing device according to claim 9, wherein the display dataincludes an existence presentation image indicating a positionrelationship between the display candidate image and the image of thepartial region to be the output target region.
 14. An informationprocessing method executed by an information processing device, themethod comprising: a specification step of specifying a transitionsource output target region and a transition destination output targetregion, in an output target region which is a partial region of anentire image, which is an image including a continuous visual field of360 degrees in at least one direction; and a visual field transitionpath determination step of automatically determining a visual fieldtransition path from the transition source output target region to thetransition destination output target region.
 15. A program that causesan information processing device to execute: a specification step ofspecifying a transition source output target region and a transitiondestination output target region, in an output target region which is apartial region of an entire image, which is an image including acontinuous visual field of 360 degrees in at least one direction; and avisual field transition path determination step of automaticallydetermining a visual field transition path from the transition sourceoutput target region to the transition destination output target region.